-- Logs begin at Fri 2022-05-27 02:29:34 UTC, end at Fri 2022-05-27 04:15:08 UTC. -- May 27 03:27:30.039877 dv-cinder-843361-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 03:27:30.048814 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 03:27:30 2022] *** May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: nodename: dv-cinder-843361-2-iscsi May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: machine: x86_64 May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: clock source: unix May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: pcre jit disabled May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: detected number of CPU cores: 4 May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: current working directory: / May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: detected binary path: /usr/bin/uwsgi-core May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: your processes number limit is 47680 May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: your memory page size is 4096 bytes May 27 03:27:30.050949 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: detected max file descriptor number: 2048 May 27 03:27:30.051745 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: lock engine: pthread robust mutexes May 27 03:27:30.051745 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: thunder lock: enabled May 27 03:27:30.051745 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 03:27:30.051745 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 03:27:30.098651 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: Python main interpreter initialized at 0x5566f021a9c0 May 27 03:27:30.098651 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: python threads support enabled May 27 03:27:30.098651 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: your server socket listen backlog is limited to 100 connections May 27 03:27:30.098651 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: your mercy for graceful operations on workers is 90 seconds May 27 03:27:30.099048 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: mapped 403077 bytes (393 KB) for 2 cores May 27 03:27:30.099127 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: *** Operational MODE: preforking *** May 27 03:27:30.099377 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: *** uWSGI is running in multiple interpreter mode *** May 27 03:27:30.099377 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: spawned uWSGI master process (pid: 80689) May 27 03:27:30.099598 dv-cinder-843361-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 03:27:30.100646 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: spawned uWSGI worker 1 (pid: 80700, cores: 1) May 27 03:27:30.100646 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: spawned uWSGI worker 2 (pid: 80701, cores: 1) May 27 03:27:30.103328 dv-cinder-843361-2-iscsi devstack@keystone.service[80689]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 03:27:35.889841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:27:35.889841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: from cryptography.utils import int_from_bytes May 27 03:27:35.890590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:27:35.890590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: from cryptography.utils import int_from_bytes May 27 03:27:35.900332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:27:35.900332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: from cryptography.utils import int_from_bytes May 27 03:27:35.900842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:27:35.900842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: from cryptography.utils import int_from_bytes May 27 03:27:35.942171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:27:35.942357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:27:35.942546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] command line args: [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:27:35.942831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:27:35.943015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ================================================================================ {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:27:35.943272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] admin_token = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.943536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] config_dir = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.943729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.943915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] config_source = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.944098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] debug = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.944300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.945797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.945797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.945797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.945797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.945797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.945797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_config_append = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.946371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.946371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_dir = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.946371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_file = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.946671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.946747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.946949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.947200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.947382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.947469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:27:35.947550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.947624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:27:35.947780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] command line args: [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:27:35.947867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:27:35.948199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ================================================================================ {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:27:35.948325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] admin_token = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] config_dir = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.948921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.949056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.949139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] config_source = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.949212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.949335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] debug = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.949448 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.949540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.950203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.950289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.950418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.950627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.950973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_config_append = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] publish_errors = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_dir = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_file = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.951964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.952991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] standard_threads = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] use_journal = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.953911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] use_json = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] use_stderr = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] use_syslog = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.954797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.954962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.955045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.955232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.955319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.955479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.955726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.955812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.955895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] publish_errors = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.955973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.956123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.956208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.956351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.956440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.956534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.956706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.956793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.956960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.957046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.957219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.957306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] standard_threads = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.957447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.957530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.957689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.957774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.957855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.957934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.958012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] use_journal = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.958166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.958246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] use_json = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.958350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] use_stderr = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.958437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.958518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] use_syslog = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.958648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.958733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:27:35.958907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.958987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.959914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.960985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.961945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.962892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.963896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.964967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.965074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.965178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.965324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.965500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.966099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.966318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.966465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.966712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.966797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.966945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967424 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.967909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.968973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.969908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.970983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.971940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.972994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.973923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.974890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.975840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.976997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.977959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.978914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.979940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.980929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.external = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.981106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.981343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.981422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.981576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.981808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.981887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.password = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] auth.token = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.982982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.983979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.external = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.984980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.password = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] auth.token = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.985992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.986921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.987934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.988992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.989978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.990960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.991933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.992997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.993974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.994953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.995970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.996126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.996209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.996373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.996454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.996709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.996816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.997983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.998064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.998233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.998344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.998492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.998744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.998879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:35.999950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.000156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.000301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.000428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.000664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.000793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.000920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.001120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.001251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.001381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.001601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.001735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.001871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.002077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.002214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.002341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.002536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.002676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.002805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.003931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.004865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.005097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.005331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.005637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.005891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.006973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.007099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.007489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.007616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.007767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.008004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.008255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.008559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.008690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.008820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.008949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.009994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.010915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.011169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.011298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.011507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.011755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.011995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.012763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.013009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.013248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.013496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.013699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.013911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.014955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.015968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.016902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.017042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.017042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.017288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.017418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.017654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.017892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.018139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.018376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.018615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.018744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.018874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] role.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] role.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.019908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.020981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.021997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.022762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.023902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.024958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.025873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.026925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] role.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] role.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.027982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.028570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.028737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.028821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.028961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.029873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.030900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.031801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.connection = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.032880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.033966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.034940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.035962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.036931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.037069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.037196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.037366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.037545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.037710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.037871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80701) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.038126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.039326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.039326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.039326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.039326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.039326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.connection = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.040962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.041278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.041436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.041673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.041916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.042158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.042407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.042648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.042898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.043145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.043393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.044152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.044420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.044690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.044943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.045191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:27:36.045369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80700) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:27:36.067379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.067687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.067825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.072840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.073087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.073289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.078044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.083876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.108308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.108608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.113149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.113387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.163812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.164391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.215025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.215342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.218949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.219187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.283598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.283881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.284021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.284233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.284450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.284689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.284823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.284956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.285141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.285368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.285593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.285811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.326167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.326437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.326437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.326775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.326924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.327991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.328127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.339481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.339700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.352287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.352547 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.352702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.352843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.352994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.353930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80700) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.354063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.354063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80701) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:27:36.371229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.371487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:27:36.372384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.372651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:27:36.372844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.373010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.373169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:27:36.373324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.373477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.373628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.373790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:27:36.373946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.374113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.374113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.374401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.374560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.385147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.385416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.385679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.385957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.386100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.386232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.386490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.386772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.394751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.395517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.398090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.398363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.398642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.398879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.398971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.399967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.400095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.400223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.400351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.400606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.400751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.400881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.401021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.401150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.401419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.401689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.427377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.428873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.429138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.429418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.430663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.430953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.435225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.435499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.435779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.436840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.437124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.437408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.443123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.443402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.444708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.444994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.447905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.448309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.448619 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.448904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.449175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.449445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.449529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.449776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.450479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.450769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.450938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.451065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.451317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.451593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.451864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.452133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.470031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.470305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.470735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.471014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.474989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.475651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.478685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.478977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.479280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.479576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.484091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.484725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.487717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.488308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.491508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.491805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.492086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.492486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.492486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.492768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.492896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.493022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.493147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.493272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.493272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.493525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.513449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.513740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.513886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.514046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.514185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.514321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.514459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.514620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.522473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.522830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.524246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.524553 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.524709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.524900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.527678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.528073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.528905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.529862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.543020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.543314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.543314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.543525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.543655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.543811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.544719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.545064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.564567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.565241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.567976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.568640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.571389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.572003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.573031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.573643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.575242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.575539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.575843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.576072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.576221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.576387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.583529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.583895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.585186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.585548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.588399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.588723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.588857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.589012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.589184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.589291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.589521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.589837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.598943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.599056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.599233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.599328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.599624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.599709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.607550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80701) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.607836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.607925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80700) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:27:36.608114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.608198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.608391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.608483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.608686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.608772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.608956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.609850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.610911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.611095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.611180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80701) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.611362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80700) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:27:36.650543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566f021a9c0 pid: 80701 (default app) May 27 03:27:36.651577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566f021a9c0 pid: 80700 (default app) May 27 03:27:36.653944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ecade-8de7-4975-bc09-bd3613f299e0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:36.654234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ecade-8de7-4975-bc09-bd3613f299e0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:36.654509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ecade-8de7-4975-bc09-bd3613f299e0 None None] PATH_INFO: `/v3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:36.658745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Fri May 27 03:27:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:45.941991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:45.942518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:45.942518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:45.947023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:45.953307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:45.953781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:45.953862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:46.024430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: INFO dbcounter [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] Registered counter for database keystone May 27 03:27:46.047249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG oslo_db.sqlalchemy.engines [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80701) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:27:46.055612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writer thread running {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:27:46.232478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG passlib.handlers.bcrypt [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80701) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:27:46.232918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG passlib.handlers.bcrypt [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80701) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:27:46.290731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:46.292352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:46.416060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 03:27:46.591039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:46.662268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: INFO dbcounter [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] Registered counter for database keystone May 27 03:27:46.681784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG oslo_db.sqlalchemy.engines [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80700) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:27:46.688246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writer thread running {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:27:46.808192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c331eb9-5828-4a73-bcb3-4add8ddefc0c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:46.813165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:46.813824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:46.814116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:46.814392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:46.822720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:46.874786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:27:46.875183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d0361-01bb-416b-855c-5012cce34a19 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:46.885902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Fri May 27 03:27:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:27:46.892715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:46.907111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6ee62-2afa-44cb-bd24-aabc3a135d40 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:46.911528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:46.912666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:46.912957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:46.913247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] PATH_INFO: `/v3/projects/1781b8350b784207b421bac2bc80c855` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:46.914764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] RBAC: Authorizing `identity:get_project(project_id=1781b8350b784207b421bac2bc80c855)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:46.969686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:27:46.970374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134edff-def7-428c-aa50-9b69cf796446 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:46.974812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Fri May 27 03:27:46 2022] GET /identity/v3/projects/1781b8350b784207b421bac2bc80c855 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:49.208637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb62f2-6e0a-43a0-89d7-a9f35397dfb2 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:49.209154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb62f2-6e0a-43a0-89d7-a9f35397dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:49.209235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb62f2-6e0a-43a0-89d7-a9f35397dfb2 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:49.213765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:27:49.220721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:49.221225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:49.221417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:49.242152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:49.243706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:49.252268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 03:27:49.275493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb62f2-6e0a-43a0-89d7-a9f35397dfb2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:49.289021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb62f2-6e0a-43a0-89d7-a9f35397dfb2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:49.294283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82f251-2766-42a1-91cf-5c71a5f274b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:49.294951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82f251-2766-42a1-91cf-5c71a5f274b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:49.295241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82f251-2766-42a1-91cf-5c71a5f274b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:49.295522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82f251-2766-42a1-91cf-5c71a5f274b9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:49.304171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82f251-2766-42a1-91cf-5c71a5f274b9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:49.306092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82f251-2766-42a1-91cf-5c71a5f274b9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:49.316360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Fri May 27 03:27:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:49.322200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:49.332275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c30016-d9a2-47cd-aed2-f17927fd8cee None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:49.336829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a68b2-1e14-4672-aefc-42cc043beec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:49.343192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a68b2-1e14-4672-aefc-42cc043beec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:49.343192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a68b2-1e14-4672-aefc-42cc043beec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:49.343192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a68b2-1e14-4672-aefc-42cc043beec0 admin admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:49.343192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a68b2-1e14-4672-aefc-42cc043beec0 admin admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:49.343192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a68b2-1e14-4672-aefc-42cc043beec0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:49.344535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Fri May 27 03:27:49 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:51.742382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8602-21a8-4497-a6c0-a74f486a431c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:51.742913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8602-21a8-4497-a6c0-a74f486a431c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:51.743317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8602-21a8-4497-a6c0-a74f486a431c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:51.746301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:51.754600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:51.755374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:51.755538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:51.775057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:51.776457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:51.780428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffeb74-3f82-4ff7-9558-7d5f1f55305d None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:51.780932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffeb74-3f82-4ff7-9558-7d5f1f55305d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:51.781373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffeb74-3f82-4ff7-9558-7d5f1f55305d None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:51.783248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:51.792372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:51.793871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:51.794262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:51.842424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG passlib.handlers.bcrypt [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80700) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:27:51.842996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG passlib.handlers.bcrypt [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80700) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:27:51.891097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:51.891741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:27:51.892773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:51.931275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:51.962426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:52.010650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:27:52.034626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:52.088278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf9f9e-6942-4f02-979c-e5d6de574322 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:52.128863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:52.129559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:52.129847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:52.130128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:52.210485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:52.212032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:52.237914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:52.325188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 7/14] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:27:52.330984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:52.331379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450d5a9-9520-4390-98e7-4fb3fb39da4d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:52.335077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:52.380280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:52.381869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:52.382235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:52.382587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:52.398057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:27:52.409596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:27:52.411368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:27:52.412567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:27:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:27:52.416921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:52.467837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3df4d-4b3f-4571-b3fa-a88b8695a7f5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:52.498951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:52.514099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:52.515252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:52.515602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:52.515962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:52.528911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:52.530908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:52.542151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5647d8-bd38-4f39-9482-cd7c6733fd2c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:52.584204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:52.584840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:52.585130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:52.585412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:52.586196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:27:52.586507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:52.588022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:52.602782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 9/16] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:27:52.608859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:52.645338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:27:52.673563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:27:52.675056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:27:52.676597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:27:52.676597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:27:52.676597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:27:52.676597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:27:52.676597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 8/17] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:27:52.681280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:52.743874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205adc2d-81f5-4003-8e1a-b788b79d16d1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:52.787561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:52.788482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:52.788808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:52.789091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:52.790905 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:52.792480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:52.794331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 10/18] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:27:52 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:27:52.800294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:52.822019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170a35-e5e4-4e2a-9840-c0cf9aad2414 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:52.868531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:52.869449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:52.869888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:52.870294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:52.871591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:27:52.871996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:52.874649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:52.895431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 9/19] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:27:52.951200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:52.963346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9fa4c-a2a5-4311-aa96-5b7b52ad80c5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:53.007120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfc611-b080-4f26-b36f-0a00bfd717bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:53.008214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc611-b080-4f26-b36f-0a00bfd717bb None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:53.008642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc611-b080-4f26-b36f-0a00bfd717bb None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:53.009011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc611-b080-4f26-b36f-0a00bfd717bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:53.010437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfc611-b080-4f26-b36f-0a00bfd717bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:53.012993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfc611-b080-4f26-b36f-0a00bfd717bb None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:53.014809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 11/20] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:27:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:27:53.021112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:53.065337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:53.174865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71540483-0e4f-4c02-bb72-1d7e8abf3de5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:53.236281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:53.237069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:53.237358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:53.237642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:53.238504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.domain.id=default, user.id=e836482fa8a8442295d2819d77066796) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:27:53.238924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:53.240455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a757c2-8dc6-43d8-b5f1-6aeb3e1e1c2b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:53.249647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 10/21] 10.222.0.58 () {56 vars in 1322 bytes} [Fri May 27 03:27:53 2022] GET /identity/v3/role_assignments?user.id=e836482fa8a8442295d2819d77066796&scope.domain.id=default&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 233 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:56.074685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcbba0-a3ad-4179-84ef-49651461b86d None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.075196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcbba0-a3ad-4179-84ef-49651461b86d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.075464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcbba0-a3ad-4179-84ef-49651461b86d None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.076937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 12/22] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:27:56.083273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0769015-8b27-4a49-be63-a5b827f84167 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.083801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0769015-8b27-4a49-be63-a5b827f84167 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.083874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0769015-8b27-4a49-be63-a5b827f84167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.107860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a0769015-8b27-4a49-be63-a5b827f84167 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:56.109144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a0769015-8b27-4a49-be63-a5b827f84167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:56.215485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 11/23] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:27:56.258119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcbba0-a3ad-4179-84ef-49651461b86d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:56.373228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca263b-3ecf-4b48-81e3-166ea74b34ca None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.373547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca263b-3ecf-4b48-81e3-166ea74b34ca None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.373801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca263b-3ecf-4b48-81e3-166ea74b34ca None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.375185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 12/24] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:27:56.381017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093dbf3-7dff-48dd-9b7a-e10b40f74100 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.381563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093dbf3-7dff-48dd-9b7a-e10b40f74100 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.381845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093dbf3-7dff-48dd-9b7a-e10b40f74100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.416115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6093dbf3-7dff-48dd-9b7a-e10b40f74100 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:56.418072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6093dbf3-7dff-48dd-9b7a-e10b40f74100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:56.422058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcbba0-a3ad-4179-84ef-49651461b86d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:56.467427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:56.509055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265701-6316-4dbe-8712-95531b213693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:56.509885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265701-6316-4dbe-8712-95531b213693 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.510253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265701-6316-4dbe-8712-95531b213693 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.510538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265701-6316-4dbe-8712-95531b213693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.536276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20265701-6316-4dbe-8712-95531b213693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:56.538211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20265701-6316-4dbe-8712-95531b213693 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:56.548739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 13/25] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:27:56.558818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378fd3-1398-487d-a6af-f3949bb020b9 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.559351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378fd3-1398-487d-a6af-f3949bb020b9 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.559847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378fd3-1398-487d-a6af-f3949bb020b9 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-20265701-6316-4dbe-8712-95531b213693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:27:56.561109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:27:56.561872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:27:56.563225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:27:56.563225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:27:56.563225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:27:56.563225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:27:56.563225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 13/26] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:27:56.564493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 14/27] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:56.568447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265701-6316-4dbe-8712-95531b213693 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:56.572910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.573210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.574462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.607779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:56.612581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:56.613915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:56.765630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 15/28] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:27:56.766903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:56.787756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265701-6316-4dbe-8712-95531b213693 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:56.862673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:56.863566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:56.863776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:56.864070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:56.864840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:27:56.865157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:56.866602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:56.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:56.923544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 14/29] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:56.926155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:56.933672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba29c-d2c3-4f4b-8d1c-99d9868f1686 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:57.015188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:57.015952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.016185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.016470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.017279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:57.018712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:57.092774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:57.130331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06410d-8ec9-4979-8f66-f619cb9f47b2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:57.170692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec76c2a-6b29-4905-8533-d70e4d67c3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:57.171747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76c2a-6b29-4905-8533-d70e4d67c3b9 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.172242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76c2a-6b29-4905-8533-d70e4d67c3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.172773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76c2a-6b29-4905-8533-d70e4d67c3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.173804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec76c2a-6b29-4905-8533-d70e4d67c3b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:57.175564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec76c2a-6b29-4905-8533-d70e4d67c3b9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:57.232399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 16/30] 10.222.0.58 () {60 vars in 1131 bytes} [Fri May 27 03:27:56 2022] POST /identity/v3/roles => generated 225 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:27:57.235000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:57.419612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 15/31] 10.222.0.58 () {60 vars in 1131 bytes} [Fri May 27 03:27:56 2022] POST /identity/v3/roles => generated 219 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:27:57.426185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117390ec-e57b-4fda-a8d3-51f1ed21fe90 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.426872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117390ec-e57b-4fda-a8d3-51f1ed21fe90 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.427364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117390ec-e57b-4fda-a8d3-51f1ed21fe90 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.429245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 16/32] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:57.430828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92305-a6bb-49c8-b546-1f7d422e380d None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.437207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92305-a6bb-49c8-b546-1f7d422e380d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.437752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92305-a6bb-49c8-b546-1f7d422e380d None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.439187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb531127-ece5-4e01-a73a-a4447587b264 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:57.439738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 17/33] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:57.441260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f3429-9e0a-4e5b-952b-426062462708 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.441786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f3429-9e0a-4e5b-952b-426062462708 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.442235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f3429-9e0a-4e5b-952b-426062462708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.486701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d34f3429-9e0a-4e5b-952b-426062462708 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:57.489122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d34f3429-9e0a-4e5b-952b-426062462708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:57.538359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:57.539567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.540086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.540626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.544825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:57.546602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:57.634639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 18/34] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:27:57.638033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ce99-988c-4677-890d-06842bcb619c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.638554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ce99-988c-4677-890d-06842bcb619c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.639067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519ce99-988c-4677-890d-06842bcb619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:57.671242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8519ce99-988c-4677-890d-06842bcb619c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:57.672618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8519ce99-988c-4677-890d-06842bcb619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:57.682232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:57.694636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 17/35] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:27:57 2022] POST /identity/v3/projects => generated 301 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:27:57.695571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:57.792840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:57.805512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 19/36] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:27:57.833300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ce99-988c-4677-890d-06842bcb619c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:57.926230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9723bb-b422-4a24-8e24-20f46bd752f8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:57.972840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519ce99-988c-4677-890d-06842bcb619c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:57.993605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f5102-98cc-4785-98ba-e3cd7004b857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:57.994857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5102-98cc-4785-98ba-e3cd7004b857 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:57.995992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5102-98cc-4785-98ba-e3cd7004b857 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:57.997279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5102-98cc-4785-98ba-e3cd7004b857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.019177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f5102-98cc-4785-98ba-e3cd7004b857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:27:58.025915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e33e6e-0630-48d1-bd8c-d657a05e8ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:58.025915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e33e6e-0630-48d1-bd8c-d657a05e8ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.025915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e33e6e-0630-48d1-bd8c-d657a05e8ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.025915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e33e6e-0630-48d1-bd8c-d657a05e8ea5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.026431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e33e6e-0630-48d1-bd8c-d657a05e8ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:58.029070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e33e6e-0630-48d1-bd8c-d657a05e8ea5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:58.030978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 20/37] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:27:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:27:58.032214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956224d6-f1b2-4833-a971-a29c862bdabe None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.032539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956224d6-f1b2-4833-a971-a29c862bdabe None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.032834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956224d6-f1b2-4833-a971-a29c862bdabe None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.034361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 21/38] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-959f5102-98cc-4785-98ba-e3cd7004b857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:27:58.035534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:27:58.036445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:27:58.037254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:27:58.038802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 18/39] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:27:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:27:58.039259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956224d6-f1b2-4833-a971-a29c862bdabe None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:58.046286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.046577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.046865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.065580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:58.067177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:58.115573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:58.139090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:58.154547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 19/40] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:27:58.157565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:58.168644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956224d6-f1b2-4833-a971-a29c862bdabe None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:58.208701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:58.209467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.209918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.210401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.211228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:58.212743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:58.279953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec4ae1-3ea2-40df-88c3-2ef534671fc1 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:58.313500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 22/41] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:27:58 2022] POST /identity/v3/projects => generated 312 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:27:58.314421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:58.315943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:58.316670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.317016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.317308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.318090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:27:58.318591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:58.320714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:58.335072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 20/42] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:27:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:58.340967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:58.417784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:58.470204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82cb5d-5ec3-446d-9d16-4e599a2d1da3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:58.489476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e29e07-f22f-49cf-9583-e47d78a77497 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:58.493250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:58.528079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:58.528802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.529091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.529375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.530146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:58.531590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:58.546975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:58.548456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.548948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.549294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.551473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:58.553374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:58.555835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 21/43] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:27:58 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:27:58.564106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:58.633366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 23/44] 10.222.0.58 () {60 vars in 1131 bytes} [Fri May 27 03:27:58 2022] POST /identity/v3/roles => generated 223 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:27:58.711243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426ae1-ead8-487c-b70a-68eb038d82ec None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:58.745082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:58.776248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263d936-e6f9-4b8b-8445-3a80e159d6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:58.779404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263d936-e6f9-4b8b-8445-3a80e159d6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:58.780091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263d936-e6f9-4b8b-8445-3a80e159d6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:58.780671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263d936-e6f9-4b8b-8445-3a80e159d6b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:58.782416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263d936-e6f9-4b8b-8445-3a80e159d6b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:58.786196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263d936-e6f9-4b8b-8445-3a80e159d6b5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:58.788579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 22/45] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:27:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:27:58.795445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:58.962549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f4b3-fc5b-4524-b703-7c85827e1381 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:59.020544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731b3e3-352c-4486-8976-5eb7a26b7e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:59.022010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731b3e3-352c-4486-8976-5eb7a26b7e29 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:59.022516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731b3e3-352c-4486-8976-5eb7a26b7e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:59.023029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731b3e3-352c-4486-8976-5eb7a26b7e29 None admin] PATH_INFO: `/v3/domains/default/users/e836482fa8a8442295d2819d77066796/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:59.026575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731b3e3-352c-4486-8976-5eb7a26b7e29 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e836482fa8a8442295d2819d77066796, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:59.029267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731b3e3-352c-4486-8976-5eb7a26b7e29 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:59.104162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 24/46] 10.222.0.58 () {58 vars in 1361 bytes} [Fri May 27 03:27:58 2022] PUT /identity/v3/domains/default/users/e836482fa8a8442295d2819d77066796/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:00.696594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f633c81-b643-4e4e-b9fc-1ea3ab2571e0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:00.697285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f633c81-b643-4e4e-b9fc-1ea3ab2571e0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:00.697285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f633c81-b643-4e4e-b9fc-1ea3ab2571e0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:00.700846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 23/47] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:00.706878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:00.707419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:00.707419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:00.726266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:00.727491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:00.758234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:00.803965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 25/48] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:00.827098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f633c81-b643-4e4e-b9fc-1ea3ab2571e0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:00.956219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f633c81-b643-4e4e-b9fc-1ea3ab2571e0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:00.995414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf4f47-d8fa-4c47-9ddb-6be7ec2b8c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:00.996231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4f47-d8fa-4c47-9ddb-6be7ec2b8c61 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:00.996545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4f47-d8fa-4c47-9ddb-6be7ec2b8c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:00.996820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf4f47-d8fa-4c47-9ddb-6be7ec2b8c61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:00.998092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4f47-d8fa-4c47-9ddb-6be7ec2b8c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:00.999551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf4f47-d8fa-4c47-9ddb-6be7ec2b8c61 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:01.001281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 24/49] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:01.007510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:01.181779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0dea8-3758-4c5b-bb9d-2cd65d22fc72 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:01.240925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3849437-f008-4958-8011-ccfd168dca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:01.241969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849437-f008-4958-8011-ccfd168dca4a None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:01.242460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849437-f008-4958-8011-ccfd168dca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:01.242934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3849437-f008-4958-8011-ccfd168dca4a None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:01.244205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3849437-f008-4958-8011-ccfd168dca4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:01.247601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3849437-f008-4958-8011-ccfd168dca4a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:01.363558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 26/50] 10.222.0.58 () {60 vars in 1140 bytes} [Fri May 27 03:28:01 2022] POST /identity/v3/projects => generated 298 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:01.367932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142f8cb-a174-4dec-9b70-71f6ccd780b8 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:01.368453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142f8cb-a174-4dec-9b70-71f6ccd780b8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:01.369037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142f8cb-a174-4dec-9b70-71f6ccd780b8 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:01.372267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 25/51] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:01.380102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:01.380405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:01.380737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:01.402085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:01.403330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:01.417806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:01.489776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 27/52] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:01.514838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142f8cb-a174-4dec-9b70-71f6ccd780b8 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:01.586375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:01.643535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142f8cb-a174-4dec-9b70-71f6ccd780b8 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:01.683614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:01.684523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:01.684821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:01.685110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:01.700089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:01.708130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:01.717321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:01.718779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 26/53] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:01.719968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:01.830651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b82d90-d40c-451e-94c4-b791534a04ed None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:01.867139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:01.867833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:01.868127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:01.868413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:01.869247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:01.869560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:01.871603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:01.882742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 28/54] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:01.889642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:02.008106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d59fb-661c-454d-9e02-3506c34178a1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:02.029405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:02.051552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:02.052661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:02.052986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:02.053510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:02.054941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:02.056597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:02.059032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 27/55] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:01 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:02.068737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:02.090075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:02.241435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb13954-b875-4bf3-ae98-8ba78e7cb163 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:02.286981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404acde9-f7fc-497f-b9c1-a736672a5ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:02.287811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404acde9-f7fc-497f-b9c1-a736672a5ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:02.288103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404acde9-f7fc-497f-b9c1-a736672a5ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:02.288390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404acde9-f7fc-497f-b9c1-a736672a5ef7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:02.289867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404acde9-f7fc-497f-b9c1-a736672a5ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:02.291390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404acde9-f7fc-497f-b9c1-a736672a5ef7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:02.293299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 29/56] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:02.300885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:02.430307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f251b-e820-4111-ab55-81982b02ec70 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:02.473608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:02.474652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:02.474961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:02.475301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:02.476778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.domain.id=default, user.id=e836482fa8a8442295d2819d77066796) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:02.477280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:02.479529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf2df1-8333-4801-a1c8-ef2eff717a44 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:02.500549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 28/57] 10.222.0.58 () {56 vars in 1322 bytes} [Fri May 27 03:28:02 2022] GET /identity/v3/role_assignments?user.id=e836482fa8a8442295d2819d77066796&scope.domain.id=default&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 530 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:03.717302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a175a-0681-4d76-a578-d8fc8000ede5 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:03.717816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a175a-0681-4d76-a578-d8fc8000ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:03.717898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a175a-0681-4d76-a578-d8fc8000ede5 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:03.719222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 30/58] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:03.724526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:03.724964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:03.725120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:03.742649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:03.743858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:03.790216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:03.813423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 29/59] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:03.837458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a175a-0681-4d76-a578-d8fc8000ede5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:03.918644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:03.949101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a175a-0681-4d76-a578-d8fc8000ede5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:03.984461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ef98e-eca5-4527-8804-3bc97a9b9ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:03.985218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef98e-eca5-4527-8804-3bc97a9b9ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:03.985508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef98e-eca5-4527-8804-3bc97a9b9ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:03.985792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ef98e-eca5-4527-8804-3bc97a9b9ea3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:03.986986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ef98e-eca5-4527-8804-3bc97a9b9ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:03.988449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ef98e-eca5-4527-8804-3bc97a9b9ea3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:03.990106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 31/60] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:03.996917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:04.095352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220a468-3553-43d1-9124-b88b1f146fc2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:04.128663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb3ec9-a584-4f7d-81ac-605b7cd96afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:04.129334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3ec9-a584-4f7d-81ac-605b7cd96afe None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:04.129623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3ec9-a584-4f7d-81ac-605b7cd96afe None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:04.129900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb3ec9-a584-4f7d-81ac-605b7cd96afe None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:04.130614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb3ec9-a584-4f7d-81ac-605b7cd96afe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:04.132306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb3ec9-a584-4f7d-81ac-605b7cd96afe None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:04.243076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 30/61] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:03 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:07.624441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991dbed-9b65-46a6-95e9-022e6fdb8d71 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:07.625032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991dbed-9b65-46a6-95e9-022e6fdb8d71 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:07.625032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991dbed-9b65-46a6-95e9-022e6fdb8d71 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:07.626610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 32/62] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:07.632429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43deb0-7941-41d1-a2a8-28842dd3e36e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:07.632924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43deb0-7941-41d1-a2a8-28842dd3e36e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:07.633074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43deb0-7941-41d1-a2a8-28842dd3e36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:07.660840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cb43deb0-7941-41d1-a2a8-28842dd3e36e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:07.662110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cb43deb0-7941-41d1-a2a8-28842dd3e36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:07.701616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:07.768560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 31/63] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:07.805799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991dbed-9b65-46a6-95e9-022e6fdb8d71 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:07.935985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980b5b5-2383-41ca-b0e4-42f18be09a8d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:07.936320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980b5b5-2383-41ca-b0e4-42f18be09a8d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:07.936599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980b5b5-2383-41ca-b0e4-42f18be09a8d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:07.938005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 32/64] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:07.950716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597f13a-4a7b-486c-af5d-01aecc97ccb3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:07.951036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597f13a-4a7b-486c-af5d-01aecc97ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:07.951294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597f13a-4a7b-486c-af5d-01aecc97ccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:07.962105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991dbed-9b65-46a6-95e9-022e6fdb8d71 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.009068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4597f13a-4a7b-486c-af5d-01aecc97ccb3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:08.010378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4597f13a-4a7b-486c-af5d-01aecc97ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:08.015426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.016250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.016559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.016844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.039047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:08.047102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:08.048026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:08.049259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 33/65] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:08.062062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.071544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.162706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 33/66] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:08.164491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2c3cb-9bbb-48ff-b76f-c0d12f86f6c8 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.165019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2c3cb-9bbb-48ff-b76f-c0d12f86f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.165512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2c3cb-9bbb-48ff-b76f-c0d12f86f6c8 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.167538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 34/67] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:08.177993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85131-acf0-4063-80b4-686acd88543d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.178597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85131-acf0-4063-80b4-686acd88543d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.179097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc85131-acf0-4063-80b4-686acd88543d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.206185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bdc85131-acf0-4063-80b4-686acd88543d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:08.207390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bdc85131-acf0-4063-80b4-686acd88543d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:08.207877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871439b-bb8a-436c-bf5d-48c0ac1876ee None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.252417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.253080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.253367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.253652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.254412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:08.254721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:08.256372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:08.265194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 34/68] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:08.266034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.292533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 35/69] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:08.292533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21bf10-c0a8-429c-9105-fc39e52ff806 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.292533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21bf10-c0a8-429c-9105-fc39e52ff806 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.292533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21bf10-c0a8-429c-9105-fc39e52ff806 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.293680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 36/70] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:08.295966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21bf10-c0a8-429c-9105-fc39e52ff806 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.331657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.350773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.394786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfddeb9-f3d0-4f84-99fb-55af4875bf85 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.427780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21bf10-c0a8-429c-9105-fc39e52ff806 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.436466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc176ab-4691-4df8-9e42-55d8dbfd412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.437309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc176ab-4691-4df8-9e42-55d8dbfd412a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.437602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc176ab-4691-4df8-9e42-55d8dbfd412a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.437890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc176ab-4691-4df8-9e42-55d8dbfd412a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.450645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc176ab-4691-4df8-9e42-55d8dbfd412a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-3bc176ab-4691-4df8-9e42-55d8dbfd412a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:08.458471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:08.459532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:08.460289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 35/71] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:08.460955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.466555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.466555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.473651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.474546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.474841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.475110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.484126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:08.485487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:08.493002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:08.494512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:08.496925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 37/72] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:08.502029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.561582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 36/73] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:08.562460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.601803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.621174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424eaa4a-2581-4881-91de-71e145482ac4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.625196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.660775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.661593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.661880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.662157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.673086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:08.678967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3b6e2-9042-49c7-b10b-bedded9bfae6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:08.681299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:08.682709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:08.683562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 38/74] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:08.684290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.723186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.723858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.724145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.724424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.725208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:08.725522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:08.726899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a8cb-36a0-4959-a800-e3176b9d10bf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:08.735470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 37/75] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:08.735470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5f52-5ffc-4003-9b08-323dc5db4e74 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.735470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5f52-5ffc-4003-9b08-323dc5db4e74 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.735470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c5f52-5ffc-4003-9b08-323dc5db4e74 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.735470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 38/76] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:08.735470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5f52-5ffc-4003-9b08-323dc5db4e74 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.800018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6aaeb-969a-4b29-aa69-fec6c8ae8c98 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.836586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.837241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.837539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.837805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.849124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c5f52-5ffc-4003-9b08-323dc5db4e74 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:08.851354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.870013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:08.893965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:08.894778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:08.895073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:08.895360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:08.908826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:08.916737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:08.917609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:08.918515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 39/77] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:08.919276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:08.929804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:08.931246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.022689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9660-f661-49ff-a04a-134cbca033aa None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.041611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 39/78] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:09.045409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.059861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.060577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.060871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.061157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.061918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:09.062233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.063633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e745d-38bd-464b-b864-66f65891f243 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.072164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 40/79] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.074659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.074952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.075241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.092120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.093571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:09.094795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:09.130136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.149712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c315f-e5ac-4ddb-915d-ebfcc0b65a22 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.166730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 41/80] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:09.180915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.195669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.196728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.197119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.198201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.199489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.201358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.203165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 40/81] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.204078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.285455 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b435143-6b3d-449c-a4c0-4e1b7743d400 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.322533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca0e86-55a8-4a3c-9235-ee84eb176147 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.326382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.327059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.327344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.327623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.328383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:09.328732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.330137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.334367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.338527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 42/82] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.339360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.358898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.360081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.360372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.360684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.361924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.363401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.365639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 41/83] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.365954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.380748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.453129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9ec11-2d90-408d-9db4-72f1cada5960 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.470505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657b81e-5dde-4219-9b39-ce6fc73f2a94 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.499239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.500129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.500425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.500745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.501988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.503457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.505139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 43/84] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.505139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.509433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.511384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.511680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.511968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.513279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.514750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.520879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 42/85] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.520879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.565657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.605085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.608022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74740a8-07b4-4af1-8bd0-756198fd0cae None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.618065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ed144-458c-48e3-822f-90aa8fa129c4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.653966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.654619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.654909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.655190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.655493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.656374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.656692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.656977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.658177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.659694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.661376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 43/86] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.668087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.731171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.732610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.768654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee316-3533-4ae6-811e-d565c58ed66e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.782738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.803995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.804831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.805223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.805516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.806367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=386db4044f5c4138b6858880d5c6bc71) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:09.806686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.809360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.814852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 44/87] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/role_assignments?user.id=386db4044f5c4138b6858880d5c6bc71&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:09.817425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.831856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:09.835883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 44/88] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:09.859068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.958017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c2004-d64b-4be5-936e-79daa00b26de None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.984636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb3492-d354-4deb-ba0c-3c2caea2a877 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.013832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.014522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.014814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.015093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.033982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.034753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.035206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.035598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.036962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.039035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.081309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:10.109353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.110960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.165615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 45/89] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:09 2022] POST /identity/v3/users => generated 289 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:10.166935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.247306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 45/90] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:10.248647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.263483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:10.331514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7091a2-68bd-4d31-9db4-471b00e81fc5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.377345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.378333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:10.378648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.379154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.379662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.407114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79d204e-cdd2-47bd-86c6-e929ab1dd1c9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.468194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.469983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.469983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.469983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.471131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.474304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.477143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 46/91] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:10.477143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.533332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.535563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.630826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:10.651345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c181a-d743-4834-8e8d-4ac04ea1a729 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.700127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.700773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 46/92] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:10.701137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.701223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.701311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.701379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.707889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.709721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.712924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 47/93] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:10.714385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.747048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:10.924787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf6468-0b44-4f45-ae0a-8e035085628f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.940308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99293-be97-4d96-9d7d-2f3d1aae3a4b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.970096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.970889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.971182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.971578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.972435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=e836482fa8a8442295d2819d77066796) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:10.972785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.974400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.984369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 47/94] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/role_assignments?user.id=e836482fa8a8442295d2819d77066796&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:10.988527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.992528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.992528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.992528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.992528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] PATH_INFO: `/v3/projects/b2dc5ee2330d4169b797edb6673614d3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:11.004741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] RBAC: Authorizing `identity:get_project(project_id=b2dc5ee2330d4169b797edb6673614d3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:11.008224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:11.012727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 48/95] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/projects/b2dc5ee2330d4169b797edb6673614d3 => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:11.020490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:11.067324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:11.222913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:11.259649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf86d6-cf8b-439f-8dc6-bc85a3525b8f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:11.316528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:11.317408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:11.317702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:11.317989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:11.319025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=b2dc5ee2330d4169b797edb6673614d3, user.id=386db4044f5c4138b6858880d5c6bc71) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:11.319346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:11.321361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ca9a3-2cce-4a1e-898a-dd9421ba3121 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:11.328621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 49/96] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:11 2022] GET /identity/v3/role_assignments?user.id=386db4044f5c4138b6858880d5c6bc71&scope.project.id=b2dc5ee2330d4169b797edb6673614d3&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:11.379628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60212bf9-13a6-45b0-9495-65c2778d6ba3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:11.477970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:11.479610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:11.480137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:11.480691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:11.482358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] RBAC: Adding query filter params (role.id=c636e98b844543518e09f390c4850083, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=386db4044f5c4138b6858880d5c6bc71) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:11.482926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:11.486038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371d0ad-14cd-4508-b7c0-78d9c4f026c3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:11.497914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 48/97] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/role_assignments?user.id=386db4044f5c4138b6858880d5c6bc71&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=c636e98b844543518e09f390c4850083 => generated 259 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:12.765924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5f45a-4ad7-409f-b85d-4e5084cd75dc None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:12.766473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5f45a-4ad7-409f-b85d-4e5084cd75dc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:12.766473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5f45a-4ad7-409f-b85d-4e5084cd75dc None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:12.768012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 50/98] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:12.783407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:12.783918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:12.784453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:12.820104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:12.822161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:13.008110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 49/99] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:13.043672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5f45a-4ad7-409f-b85d-4e5084cd75dc None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.213773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5f45a-4ad7-409f-b85d-4e5084cd75dc None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.259586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.260445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.260754 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.261052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.280390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:13.288422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:13.290664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:13.293381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 51/100] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:13.298680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.317506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:13.437748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f6aea-6a82-4038-af3e-cffcb631d1df None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.497437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.498167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.498452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.498740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.499531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:13.499848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:13.501305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:13.511765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 50/101] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:13.517537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.528334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:13.653853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cee990-09de-4746-aa1a-80f54d8398a7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.699401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.700482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.700802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.701086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.702458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:13.704011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:13.705848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 52/102] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:13.711578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.831262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:13.864757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50d7ee-7743-4c9a-bced-739999717587 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.927268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6e5f-8ae7-4c1d-b633-ac013b340363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.928159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6e5f-8ae7-4c1d-b633-ac013b340363 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.928452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6e5f-8ae7-4c1d-b633-ac013b340363 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.928765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6e5f-8ae7-4c1d-b633-ac013b340363 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.933395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea6e5f-8ae7-4c1d-b633-ac013b340363 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:13.934884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea6e5f-8ae7-4c1d-b633-ac013b340363 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:13.937007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 51/103] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:13.942510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:14.048105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:14.075780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc45fc3-2af1-4ca9-ab58-259bf595f9f7 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.076113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc45fc3-2af1-4ca9-ab58-259bf595f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.076356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc45fc3-2af1-4ca9-ab58-259bf595f9f7 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.077923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 52/104] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:14.090296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01f06e-56bf-4b47-a3cb-a135993479d3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:14.095011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.095307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.095598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.129483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:14.130965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:14.180186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5416d2-b1cd-424c-b5c4-ea43fd9610d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:14.181098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5416d2-b1cd-424c-b5c4-ea43fd9610d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.181391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5416d2-b1cd-424c-b5c4-ea43fd9610d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.181679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5416d2-b1cd-424c-b5c4-ea43fd9610d3 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081/users/386db4044f5c4138b6858880d5c6bc71/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.185928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5416d2-b1cd-424c-b5c4-ea43fd9610d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=059d0856d2f94b658054f1a53c494081, user_id=386db4044f5c4138b6858880d5c6bc71, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:14.187456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5416d2-b1cd-424c-b5c4-ea43fd9610d3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:14.262304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 53/105] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:14.278410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 53/106] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:13 2022] PUT /identity/v3/projects/059d0856d2f94b658054f1a53c494081/users/386db4044f5c4138b6858880d5c6bc71/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:14.312120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:14.387319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f162b09-1505-4c67-9f2f-bc39a108819e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.387690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f162b09-1505-4c67-9f2f-bc39a108819e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.388071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f162b09-1505-4c67-9f2f-bc39a108819e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.389922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 54/107] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:14.396520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.397474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.397474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.418354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:14.419674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:14.443173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:14.508206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 55/108] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:14.531010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdbbb9-fcb7-45b9-92a7-4bc260d26478 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:14.533775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:14.608915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:14.645914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:14.646811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.647054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.647340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.654054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1d96e-71d8-4bc3-afaf-914e9a903719 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:14.686824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:14.694918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:14.695816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.696199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.700534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.700534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:14.700534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:14.701518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:14.702466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:14.703306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:14.703878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 56/109] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:14.705450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 54/110] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:14.710176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:14.711857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:14.818371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfc30d-5653-462d-885c-dda86b6d5599 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:14.852964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:14.860531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:14.860531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.860531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.860531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.861054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:14.862955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:14.907370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb229b8-16af-4382-958b-18b15401848f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:14.957254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b139a36-4774-428a-802b-53384776d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:14.957896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b139a36-4774-428a-802b-53384776d552 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.958208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b139a36-4774-428a-802b-53384776d552 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.958486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b139a36-4774-428a-802b-53384776d552 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.959282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b139a36-4774-428a-802b-53384776d552 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:14.959590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b139a36-4774-428a-802b-53384776d552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:14.961289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b139a36-4774-428a-802b-53384776d552 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:14.963145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:14.981791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 57/111] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:28:14 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:14.982835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 55/112] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:14.989483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.145918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73706e5-1a41-42b0-997b-05e6265a2ca8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:15.193303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8243d2-ae81-4b62-b6b9-f9a8d6f70486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:15.194236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8243d2-ae81-4b62-b6b9-f9a8d6f70486 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.194532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8243d2-ae81-4b62-b6b9-f9a8d6f70486 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.194815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8243d2-ae81-4b62-b6b9-f9a8d6f70486 None admin] PATH_INFO: `/v3/users/dde9c5630eb64351a7888ba1a80aa0ab` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.220644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8243d2-ae81-4b62-b6b9-f9a8d6f70486 None admin] RBAC: Authorizing `identity:get_user(user_id=dde9c5630eb64351a7888ba1a80aa0ab)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:15.222090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8243d2-ae81-4b62-b6b9-f9a8d6f70486 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:15.224169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 58/113] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:14 2022] GET /identity/v3/users/dde9c5630eb64351a7888ba1a80aa0ab => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:15.231349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b139a36-4774-428a-802b-53384776d552 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.294603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f196d-d339-464b-9966-1521acd7e866 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.294929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f196d-d339-464b-9966-1521acd7e866 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.295181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f196d-d339-464b-9966-1521acd7e866 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.296782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 59/114] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:15.306125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776deab-6887-4af5-b730-a50dff40711e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.306425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776deab-6887-4af5-b730-a50dff40711e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.306763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776deab-6887-4af5-b730-a50dff40711e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.308487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 60/115] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:15.314010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0da2c-d43b-4cf4-baca-a302be06e761 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.314303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0da2c-d43b-4cf4-baca-a302be06e761 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.314591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0da2c-d43b-4cf4-baca-a302be06e761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.340235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0fe0da2c-d43b-4cf4-baca-a302be06e761 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:15.341539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0fe0da2c-d43b-4cf4-baca-a302be06e761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:15.393666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b139a36-4774-428a-802b-53384776d552 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:15.396918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:15.438035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 61/116] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 03:28:15.439707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d4247-845b-4cbc-a491-1b5e19ee235f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.440216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d4247-845b-4cbc-a491-1b5e19ee235f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.440854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d4247-845b-4cbc-a491-1b5e19ee235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.449897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc573cb4-8b56-47fc-805c-73ac3ee3e8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:15.451043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc573cb4-8b56-47fc-805c-73ac3ee3e8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.451565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc573cb4-8b56-47fc-805c-73ac3ee3e8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.452080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc573cb4-8b56-47fc-805c-73ac3ee3e8f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.460574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f39d4247-845b-4cbc-a491-1b5e19ee235f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:15.462134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f39d4247-845b-4cbc-a491-1b5e19ee235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:15.548015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 62/117] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:15.549683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857287-64f3-47c6-8c80-497e169c82ca None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.549987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857287-64f3-47c6-8c80-497e169c82ca None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.550338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857287-64f3-47c6-8c80-497e169c82ca None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.552052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 63/118] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:15.557080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857287-64f3-47c6-8c80-497e169c82ca None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.562938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:15.572270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc573cb4-8b56-47fc-805c-73ac3ee3e8f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:15.576247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc573cb4-8b56-47fc-805c-73ac3ee3e8f0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:15.688963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857287-64f3-47c6-8c80-497e169c82ca None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:15.691073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:15.714732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 56/119] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:15.716019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.716319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.716685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.737165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:15.738052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.738394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.738680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.739520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:15.742796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:15.752105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:15.761791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:15.763707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 64/120] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:15.765606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.874624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 57/121] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:15.875480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.926568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:15.951194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68427afe-b9a5-4a7d-8eae-3d2845c7afbb None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.011755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.013810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.014252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.014834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.028146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b272c-e382-4c16-a292-8a386ad713d1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.030283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:16.040252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:16.041183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:16.042286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:16.045021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 65/122] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:16.046283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.077116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.078429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.078837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.080105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.082289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.084100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.085638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.088832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 58/123] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:15 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.090048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.202642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53346dee-d7c1-412d-b758-8dd9bd24fb01 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.212358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.220544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec6cb1-6fc8-4894-9cb6-2cdf4830d7b0 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.251880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.253051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.253498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.254192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.255501 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:16.255822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.257258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.259678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.261919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.262430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.262952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.267812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 66/124] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.268944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.275262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:16.284123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:16.285292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:16.286066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:16.288693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 59/125] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:16.289798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.387875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.405958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6712f5-eb4e-4cab-bc61-f09d15a7e0c6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.420181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54613-c3ad-4076-b5e8-565afa0da398 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.447141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.448817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.449144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.449432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.450323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:16.450645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.452196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.459707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.460712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.461204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.461629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 67/126] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.462134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.462236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.464628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] RBAC: Adding query filter params (role.id=4a579dca3f24408ab6efbd3f9f1e1ed6, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=dde9c5630eb64351a7888ba1a80aa0ab) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:16.465172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.467224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.473789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 60/127] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/role_assignments?user.id=dde9c5630eb64351a7888ba1a80aa0ab&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.474996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.499243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.588667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799c149-d37f-4d71-b437-7fce79e3713f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.602819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500a9b3-def7-4553-b911-2cf9db356a5f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.636993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.638311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.638808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.639277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.641069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.643022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.645159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 68/128] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.646292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.647548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.648288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.648615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.648901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.649672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:16.650242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.652539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.661969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 61/129] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.662868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.673344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.760158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.780223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a341e5-5d83-40bb-ac4e-4af2dcd3e2f4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.782854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04177a-ce2d-4895-8da5-057d71dcf974 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.821545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.822821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.823309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.823981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.824800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.825614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.826036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.826722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.827474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.827830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.829780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 69/130] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.830234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.831067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.831804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.833945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 62/131] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.840331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.946358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f19ee-da26-4734-8235-ee29a1d293e1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.965776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f647-e00d-4ea8-890a-90330104b6aa None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.981188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:16.990586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.991563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.991856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.992144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.993706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.996692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.998916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 70/132] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:16.999831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:17.011975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cac97-8164-4352-b128-70feefea305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:17.012929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cac97-8164-4352-b128-70feefea305b None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.013226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cac97-8164-4352-b128-70feefea305b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.013515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cac97-8164-4352-b128-70feefea305b None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081/users/386db4044f5c4138b6858880d5c6bc71/roles/c636e98b844543518e09f390c4850083` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.016172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cac97-8164-4352-b128-70feefea305b None admin] RBAC: Authorizing `identity:create_grant(project_id=059d0856d2f94b658054f1a53c494081, user_id=386db4044f5c4138b6858880d5c6bc71, role_id=c636e98b844543518e09f390c4850083)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:17.017752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cac97-8164-4352-b128-70feefea305b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:17.139629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9745ae6-f66b-405c-9d59-af97160c8eb2 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:17.148702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 63/133] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:16 2022] PUT /identity/v3/projects/059d0856d2f94b658054f1a53c494081/users/386db4044f5c4138b6858880d5c6bc71/roles/c636e98b844543518e09f390c4850083 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:28:17.149651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cac97-8164-4352-b128-70feefea305b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:17.179216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:17.205348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:17.206786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.207273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.207741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.210125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:17.212845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:17.216113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 71/134] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:17.225594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:17.305006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:17.407287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326ee68-a2a0-4f4b-a66f-ecc3c90a4eb1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:17.471468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5af1c7-559b-42be-86b4-a63a0f286259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:17.472455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af1c7-559b-42be-86b4-a63a0f286259 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.472770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af1c7-559b-42be-86b4-a63a0f286259 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.473101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5af1c7-559b-42be-86b4-a63a0f286259 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081/users/e836482fa8a8442295d2819d77066796/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.475742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5af1c7-559b-42be-86b4-a63a0f286259 None admin] RBAC: Authorizing `identity:create_grant(project_id=059d0856d2f94b658054f1a53c494081, user_id=e836482fa8a8442295d2819d77066796, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:17.477262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5af1c7-559b-42be-86b4-a63a0f286259 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:17.494698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cac97-8164-4352-b128-70feefea305b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:17.549952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 72/135] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:17 2022] PUT /identity/v3/projects/059d0856d2f94b658054f1a53c494081/users/e836482fa8a8442295d2819d77066796/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:17.556085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2283a07-61ff-478b-afc2-dd960a5d9d05 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.556357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2283a07-61ff-478b-afc2-dd960a5d9d05 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.557555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2283a07-61ff-478b-afc2-dd960a5d9d05 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.565245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 73/136] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:17.566405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:17.567301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.567593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.567876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] PATH_INFO: `/v3/projects/b2dc5ee2330d4169b797edb6673614d3` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.569680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] RBAC: Authorizing `identity:get_project(project_id=b2dc5ee2330d4169b797edb6673614d3)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:17.571383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:17.573981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.574232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 64/137] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:17 2022] GET /identity/v3/projects/b2dc5ee2330d4169b797edb6673614d3 => generated 312 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:17.574784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.575078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.582383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:17.625806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:17.628796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:17.763830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:17.796591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 74/138] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:17.811253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25521e5-f8b7-42d4-bda2-e671a4832c0a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:17.852793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:17.865667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37125d8-8e59-4e2d-8622-b75433722ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:17.867548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37125d8-8e59-4e2d-8622-b75433722ff0 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.867662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37125d8-8e59-4e2d-8622-b75433722ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.868167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37125d8-8e59-4e2d-8622-b75433722ff0 None admin] PATH_INFO: `/v3/projects/b2dc5ee2330d4169b797edb6673614d3/users/386db4044f5c4138b6858880d5c6bc71/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.874236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:17.877414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37125d8-8e59-4e2d-8622-b75433722ff0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2dc5ee2330d4169b797edb6673614d3, user_id=386db4044f5c4138b6858880d5c6bc71, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:17.879050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37125d8-8e59-4e2d-8622-b75433722ff0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:17.957778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 65/139] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:17 2022] PUT /identity/v3/projects/b2dc5ee2330d4169b797edb6673614d3/users/386db4044f5c4138b6858880d5c6bc71/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:18.019355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf19a7a-2c83-4be5-8b69-f1c38001837c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.019951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf19a7a-2c83-4be5-8b69-f1c38001837c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.019951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf19a7a-2c83-4be5-8b69-f1c38001837c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.021533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 66/140] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:18.030210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8e4a6-4880-429e-a9f5-8783cb92ceb1 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.035121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.035460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.035702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.084140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:18.085346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:18.099747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.101591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.102374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.103117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.126943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:18.143439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:18.145865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:18.147380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:18.151505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 75/141] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:18.160941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.249413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 67/142] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:18.299803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:18.305663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.406267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce33d9-a462-476e-9f32-8b663d022f61 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.462109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:18.485314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.486024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.486299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.486578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.487350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:18.487663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.492531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.500025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 76/143] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:18.505802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.515121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff4a537-e649-46e9-8628-e6b4d9ce532b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.604721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.605551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.605838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.606116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.612115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.613675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.615421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 68/144] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:18.621968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.639023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10896a0-b525-4f6f-a7c6-f309dff22b35 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.675623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:18.683780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.685478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.685774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.686058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.687804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.691390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.693913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 77/145] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:18 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:18.711072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.827085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458b73-f105-4d8b-ad0c-7a4ae517b1a7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.849695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb4eaa-d346-45ac-9143-ab7e20358794 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.870006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.870006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.870006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.870006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.955859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:18.960558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.962114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.973320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.974140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.974444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.974729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.975482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.976998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.990715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:19.079680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 69/146] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:18 2022] POST /identity/v3/users => generated 283 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:28:19.087579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 78/147] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:19.098680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:19.284596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760166d0-6cdf-4342-852a-fa93d553694b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:19.350708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e634b5-4794-4ff3-966f-4c3b7917c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:19.351739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e634b5-4794-4ff3-966f-4c3b7917c976 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:19.352047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e634b5-4794-4ff3-966f-4c3b7917c976 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:19.352337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e634b5-4794-4ff3-966f-4c3b7917c976 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:19.354215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e634b5-4794-4ff3-966f-4c3b7917c976 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:19.355917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e634b5-4794-4ff3-966f-4c3b7917c976 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:19.358283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 70/148] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:19 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:19.364685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:19.503983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:19.510371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18417b-856e-4cde-9937-7c1ed54b29a3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:19.557190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:19.558009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:19.558417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:19.558701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:19.559556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=386db4044f5c4138b6858880d5c6bc71) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:19.559873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:19.561389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09cf4-a985-4758-881d-ad813b1923e9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:19.569159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 79/149] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:19 2022] GET /identity/v3/role_assignments?user.id=386db4044f5c4138b6858880d5c6bc71&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:20.117731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a081831-5777-4704-b47d-c961ea55b294 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.118290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a081831-5777-4704-b47d-c961ea55b294 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.118290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a081831-5777-4704-b47d-c961ea55b294 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.119914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 71/150] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:20.126150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.126672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.127129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.151050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:20.152549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:20.250092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 80/151] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:20.273380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a081831-5777-4704-b47d-c961ea55b294 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:20.364311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:20.419142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a081831-5777-4704-b47d-c961ea55b294 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:20.469636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593b99d-9b65-4b24-9208-2eb8e4917a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:20.470515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593b99d-9b65-4b24-9208-2eb8e4917a94 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.470822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593b99d-9b65-4b24-9208-2eb8e4917a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.471089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593b99d-9b65-4b24-9208-2eb8e4917a94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.491837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593b99d-9b65-4b24-9208-2eb8e4917a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-0593b99d-9b65-4b24-9208-2eb8e4917a94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:20.499865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:20.500808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:20.501731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:20.505582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 72/152] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:20.511979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:20.639492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:20.649535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a74f7-b992-4e9a-8119-4fa564b66304 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:20.675319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ce598-706a-4a80-ba01-ea2a77f4a7c0 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.675754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ce598-706a-4a80-ba01-ea2a77f4a7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.676247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ce598-706a-4a80-ba01-ea2a77f4a7c0 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.678955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 73/153] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:20.687618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.687927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.688192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.721447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:20.722491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.722990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.723480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.724646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] RBAC: Adding query filter params (name=reader) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:20.725179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:20.726896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:20.730691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:20.731656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:20.739318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 81/154] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:20.751155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:20.852061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 74/155] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:20.883296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:20.964323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf24bd-0d78-437b-87f3-9ad51b4a2ded None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:21.008911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:21.054701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:21.055646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.055926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.056198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] PATH_INFO: `/v3/users/dde9c5630eb64351a7888ba1a80aa0ab` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.065520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] RBAC: Authorizing `identity:get_user(user_id=dde9c5630eb64351a7888ba1a80aa0ab)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:21.067062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:21.069223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 82/156] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:20 2022] GET /identity/v3/users/dde9c5630eb64351a7888ba1a80aa0ab => generated 289 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:21.085035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:21.138829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14144b8b-d882-4cdf-943a-8eb15a9afcdf None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:21.201115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:21.221431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:21.222174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.222470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.222749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.242182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:21.260465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:21.261931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:21.263461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 75/157] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:21.270634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:21.294824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38adde-2b6d-49f2-9800-3539f5e8a1cb None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:21.358471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:21.359373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.359654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.359930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.365486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:21.366996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:21.369378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 83/158] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:21 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:21.386073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:21.462779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd431c3-146d-440b-affd-ed83df2ad7be None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:21.482036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:21.540004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:21.554733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:21.555121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546dbd6-e759-465c-9e81-c0d930287f11 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:21.556052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.557109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.557577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.558606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:21.559142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:21.569350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:21.586140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 76/159] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:21.592425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:21.621318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cad32-460d-4d34-9fc7-034c31010917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:21.622230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cad32-460d-4d34-9fc7-034c31010917 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.622509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cad32-460d-4d34-9fc7-034c31010917 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.622781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cad32-460d-4d34-9fc7-034c31010917 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081/users/dde9c5630eb64351a7888ba1a80aa0ab/roles/4a579dca3f24408ab6efbd3f9f1e1ed6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.625918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cad32-460d-4d34-9fc7-034c31010917 None admin] RBAC: Authorizing `identity:create_grant(project_id=059d0856d2f94b658054f1a53c494081, user_id=dde9c5630eb64351a7888ba1a80aa0ab, role_id=4a579dca3f24408ab6efbd3f9f1e1ed6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:21.627402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cad32-460d-4d34-9fc7-034c31010917 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:21.703950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 84/160] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:21 2022] PUT /identity/v3/projects/059d0856d2f94b658054f1a53c494081/users/dde9c5630eb64351a7888ba1a80aa0ab/roles/4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:21.775203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96430ea5-3faf-4322-821b-03817d1b81c3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:21.831290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:21.832782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.833389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.833857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.835766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:21.841736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:21.844318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 77/161] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:21 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:21.849301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d78a-3329-44a1-8e1c-20b26d372f52 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.849589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d78a-3329-44a1-8e1c-20b26d372f52 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.849867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d78a-3329-44a1-8e1c-20b26d372f52 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.850308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:21.851397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 85/162] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:21.858075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:21.858412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:21.858631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:21.906837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:21.908118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:21.955077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:22.049600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 86/163] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:22.079810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789221c7-5f63-4f2b-a905-dc06b81a5c05 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.110953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.131573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.132291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.132604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.132893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.278544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.280134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.288929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:22.331483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c72421-b4e4-4f7a-9e57-18962ecb468b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.370121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:22.395338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.400670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.400946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.401222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.425116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:22.433466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:22.434378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:22.435671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 87/164] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:22.445043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.528938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 78/165] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:22.536001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.632887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5415b67-808b-475c-86f6-bd9f0077462b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.749566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.750635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.751101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.751550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.752867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:22.753383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.756031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.782449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 88/166] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:22.791474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.811836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:22.838501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af8fb-49a3-4f10-9ae3-3db8ec62160d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.937875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.940000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.940730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.941406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.944023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.947116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.958243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 79/167] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:22.979889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.009371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:23.011343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7f1ae-95ee-41e5-b704-6fd4ab137693 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.054094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e512e0-0e9e-4dee-950f-f2a085a4a65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.054557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e512e0-0e9e-4dee-950f-f2a085a4a65a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.054936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e512e0-0e9e-4dee-950f-f2a085a4a65a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.055331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e512e0-0e9e-4dee-950f-f2a085a4a65a None admin] PATH_INFO: `/v3/users/e836482fa8a8442295d2819d77066796` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.057240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e512e0-0e9e-4dee-950f-f2a085a4a65a None admin] RBAC: Authorizing `identity:get_user(user_id=e836482fa8a8442295d2819d77066796)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.059257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e512e0-0e9e-4dee-950f-f2a085a4a65a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.061612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 89/168] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/users/e836482fa8a8442295d2819d77066796 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:23.069278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19a8fd-8680-4d96-87d9-28f55e4e252c None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.069278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19a8fd-8680-4d96-87d9-28f55e4e252c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.069278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19a8fd-8680-4d96-87d9-28f55e4e252c None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.069278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 90/169] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:23.071415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19a8fd-8680-4d96-87d9-28f55e4e252c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.246693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:23.276445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828f615-dccb-408c-9c6a-9ecc8e0fdc1f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.285642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19a8fd-8680-4d96-87d9-28f55e4e252c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.330477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.331204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.331666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.331991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.332982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] RBAC: Adding query filter params (role.id=c636e98b844543518e09f390c4850083, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=386db4044f5c4138b6858880d5c6bc71) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:23.333410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.335159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728b73c-5e69-4f84-8a1e-65f8e008d404 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.344060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.344785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.345071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.345348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.345827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 80/170] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/role_assignments?user.id=386db4044f5c4138b6858880d5c6bc71&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=c636e98b844543518e09f390c4850083 => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:23.348031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.348462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.348939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.388081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:23.389547 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:23.483011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.484544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.531216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:23.541600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 81/171] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:23.575773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:23.583946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.613157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 91/172] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:23.622703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.796067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a323991-a141-442f-9fa9-05d40e6a3774 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.843200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28135e5-49c2-45a4-aa36-c000f2b86279 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.863810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.864644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.864932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.865215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.886536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:23.895024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:23.896802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:23.898802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:23.901410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 82/173] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:23.905362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.905362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.905362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.905362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.906335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.906775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.908591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.910713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 92/174] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:23.918394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.997688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:24.031081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:24.121848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84feac-89a8-430d-a5e9-dfe8b71babec None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:24.138771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc73b-b8ae-4d90-bcf2-6b862721a87e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:24.184611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:24.185304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.185592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.185870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.186958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=e836482fa8a8442295d2819d77066796) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:24.187292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:24.189286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:24.198248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:24.198696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 93/175] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/role_assignments?user.id=e836482fa8a8442295d2819d77066796&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:24.199675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.200161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.200674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.201648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:24.202185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:24.203855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d621dc-38fd-45c7-9997-0e0c3571143c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:24.220267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 83/176] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:24.226519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:24.235766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8bdbe-5da5-44ff-846a-1e2eb873cea9 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.236075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8bdbe-5da5-44ff-846a-1e2eb873cea9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.236333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8bdbe-5da5-44ff-846a-1e2eb873cea9 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.237902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 84/177] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:24.243477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.244356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.244924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.274199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:24.275981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:24.389780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e1ebe-2215-4a11-960b-cda4765a5996 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:24.405399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:24.457331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:24.458253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.458794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.458897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 85/178] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:24.459304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] PATH_INFO: `/v3/users/386db4044f5c4138b6858880d5c6bc71` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.460276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] RBAC: Authorizing `identity:get_user(user_id=386db4044f5c4138b6858880d5c6bc71)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:24.461872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:24.463837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 94/179] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:24 2022] GET /identity/v3/users/386db4044f5c4138b6858880d5c6bc71 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:24.469749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:24.499378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:24.584995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:24.641048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4c6a2-8caa-4a55-a4fe-874bbff6183a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:24.681836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:24.682675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.683148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.683431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.686624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e0007-a11a-4b7b-9d3e-f96de394e70b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:24.698239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:24.706582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:24.707728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:24.708740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:24.711388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 95/180] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:24.722367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:24.781815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:24.786218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c401ee2-fa8b-4575-876b-d3b446f6977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:24.786218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c401ee2-fa8b-4575-876b-d3b446f6977b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.786218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c401ee2-fa8b-4575-876b-d3b446f6977b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.786218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c401ee2-fa8b-4575-876b-d3b446f6977b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.862521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c34db-8380-4db2-a564-0c0e81963d52 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:24.921370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:24.922502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.922942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.923447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.924604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:24.925089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:24.926889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e910fd-05b1-463b-8ca4-edaac578a29f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:24.938019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 96/181] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:24.940137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf510-725d-49a3-9e06-b6cfd1ee9b5b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:24.940729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf510-725d-49a3-9e06-b6cfd1ee9b5b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:24.942234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dccf510-725d-49a3-9e06-b6cfd1ee9b5b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:24.944661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 97/182] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:24.948879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf510-725d-49a3-9e06-b6cfd1ee9b5b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:24.971589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c401ee2-fa8b-4575-876b-d3b446f6977b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:24.973138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c401ee2-fa8b-4575-876b-d3b446f6977b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.057195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.070574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.083149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dccf510-725d-49a3-9e06-b6cfd1ee9b5b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.121635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53263a8-edcd-4b1a-b47d-96f346e753b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.122530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53263a8-edcd-4b1a-b47d-96f346e753b3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.122921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53263a8-edcd-4b1a-b47d-96f346e753b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.123204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53263a8-edcd-4b1a-b47d-96f346e753b3 None admin] PATH_INFO: `/v3/users/213f04f9218f43408f0e565f33093ea7` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.135291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 86/183] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:25.136738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.137037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.137321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.141968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53263a8-edcd-4b1a-b47d-96f346e753b3 None admin] RBAC: Authorizing `identity:get_user(user_id=213f04f9218f43408f0e565f33093ea7)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.144755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53263a8-edcd-4b1a-b47d-96f346e753b3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.147906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 98/184] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:24 2022] GET /identity/v3/users/213f04f9218f43408f0e565f33093ea7 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:25.154684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55c55-3bbf-4804-995a-414eb9abaefb None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.155161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55c55-3bbf-4804-995a-414eb9abaefb None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.155438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55c55-3bbf-4804-995a-414eb9abaefb None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.156345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:25.157250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 99/185] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:25.162084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:25.162348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e55c55-3bbf-4804-995a-414eb9abaefb None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.252531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 87/186] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:25.253328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.281650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e55c55-3bbf-4804-995a-414eb9abaefb None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.326897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da13f9-c86b-4c64-b191-599f562d3569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.328069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da13f9-c86b-4c64-b191-599f562d3569 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.328492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da13f9-c86b-4c64-b191-599f562d3569 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.328924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da13f9-c86b-4c64-b191-599f562d3569 None admin] PATH_INFO: `/v3/projects/b2dc5ee2330d4169b797edb6673614d3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.330234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da13f9-c86b-4c64-b191-599f562d3569 None admin] RBAC: Authorizing `identity:get_project(project_id=b2dc5ee2330d4169b797edb6673614d3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.331835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da13f9-c86b-4c64-b191-599f562d3569 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.334015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 100/187] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/projects/b2dc5ee2330d4169b797edb6673614d3 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:25.335350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.335639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.335913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.355379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:25.357167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:25.362042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.375957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.384053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af14d9-96ea-4ab6-9293-fb810bad3e29 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.426029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.427456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.427982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.428406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.440558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 101/188] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:25.442010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.520904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.522666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.570773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4658c-cc0f-4a04-9739-5cbc284cc92f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.610654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.618121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.619240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.619652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.619931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.621179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.622739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.625967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 102/189] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:25.625967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.637084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 88/190] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:25.638236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.648910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.745018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ce246-ec12-42c1-a006-1afc2a3c418a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.756369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4519470-b31b-490f-b63b-d9112c4da82f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.787317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.788036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.788577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.788713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.794485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.795556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.795837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.796116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.796990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=b2dc5ee2330d4169b797edb6673614d3, user.id=386db4044f5c4138b6858880d5c6bc71) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:25.797673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.799586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:25.799705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.807934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 103/191] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/role_assignments?user.id=386db4044f5c4138b6858880d5c6bc71&scope.project.id=b2dc5ee2330d4169b797edb6673614d3&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:25.808827 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:25.809829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:25.810805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:25.811597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 89/192] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:25.812232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.880127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.928982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:25.952811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3492b1a-a713-4d08-ad60-4f3f59d68d3b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.955598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9d891-b2ca-479a-914c-44e57d365ff2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.995085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.995803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.996091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.996563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.997860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.997972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62ab5e-4a3f-4fb6-8ffa-1fffdce83d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.999332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62ab5e-4a3f-4fb6-8ffa-1fffdce83d2d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.999332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62ab5e-4a3f-4fb6-8ffa-1fffdce83d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.999332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62ab5e-4a3f-4fb6-8ffa-1fffdce83d2d None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.000701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62ab5e-4a3f-4fb6-8ffa-1fffdce83d2d None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:26.000807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:26.002271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62ab5e-4a3f-4fb6-8ffa-1fffdce83d2d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:26.013075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 90/193] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:26.013075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3b4c-b445-42f3-85d2-4734889106cc None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.013075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3b4c-b445-42f3-85d2-4734889106cc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.013075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db3b4c-b445-42f3-85d2-4734889106cc None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.013075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 91/194] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:26.013075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3b4c-b445-42f3-85d2-4734889106cc None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.127128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 104/195] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:25 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:26.127889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.140357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db3b4c-b445-42f3-85d2-4734889106cc None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:26.153768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:26.182265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:26.182958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.183243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.183520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.184289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:26.184626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:26.186049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b582b6-8dfb-4695-96c3-c07818e986e3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:26.195915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 92/196] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:26.198214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.198513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.198798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.218487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:26.219738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:26.260856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e45e8c-19b4-43a2-8d0d-44811c502208 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:26.306207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:26.307054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 93/197] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:26.307163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.307163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.307321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.307403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.307403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=213f04f9218f43408f0e565f33093ea7) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:26.307403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:26.307403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:26.308589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:26.314758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 105/198] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/role_assignments?user.id=213f04f9218f43408f0e565f33093ea7&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:26.325826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.469639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a716c4-740a-4673-a2fa-c51d38b9d1c5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:26.472521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:26.514284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f170b-0431-4a45-8ee0-9566a7ae5eb4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:26.519507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:26.520538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.520718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.521145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] PATH_INFO: `/v3/users/213f04f9218f43408f0e565f33093ea7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.522323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] RBAC: Authorizing `identity:get_user(user_id=213f04f9218f43408f0e565f33093ea7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:26.523702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:26.526352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 94/199] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/users/213f04f9218f43408f0e565f33093ea7 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:26.532477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.569733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:26.570540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.570821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.571095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.583042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:26.591063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:26.592001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:26.592918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:26.596613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 106/200] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:26.599441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.671107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:26.683106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30579cc2-106b-4d20-93c4-67158a2345cf None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:26.747309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247df32-cfbb-4c80-b281-bcc583373744 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:26.757057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:26.757726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.758014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.758290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.816600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:26.817285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:26.817567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:26.817838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:26.818597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:26.818904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:26.820311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:26.835643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 107/201] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:26.837086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:26.843591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:26.899213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:26.901032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:27.055517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3b92e-7795-4d42-ad87-3119d80f0dc9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:27.081100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:27.083076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 95/202] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:27.097252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:27.115814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:27.117116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:27.118025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:27.119392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] PATH_INFO: `/v3/users/dde9c5630eb64351a7888ba1a80aa0ab` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:27.122490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] RBAC: Authorizing `identity:get_user(user_id=dde9c5630eb64351a7888ba1a80aa0ab)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:27.124199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:27.130835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 108/203] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:26 2022] GET /identity/v3/users/dde9c5630eb64351a7888ba1a80aa0ab => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:27.137252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:27.212479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:27.249723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a025c5f-c172-41be-bd90-edd77c6c1ec6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:27.302975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:27.304239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:27.304770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:27.305237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:27.306823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:27.308644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:27.310607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 96/204] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:27 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:27.321640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:27.372793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7519eef-ce97-4bc9-b99a-10afb6fb9121 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:27.440922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:27.441583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:27.441867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:27.442144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:27.516582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61de49c-44d5-46a2-a2e3-f4736db343df None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:27.541097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:27.562205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:27.562205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:27.595066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:27.596620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:27.596976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:27.597323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:27.598231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] RBAC: Adding query filter params (role.id=c636e98b844543518e09f390c4850083, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=213f04f9218f43408f0e565f33093ea7) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:27.598611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:27.600163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:27.609919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 97/205] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:27 2022] GET /identity/v3/role_assignments?user.id=213f04f9218f43408f0e565f33093ea7&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=c636e98b844543518e09f390c4850083 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:27.716883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 109/206] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:27.722573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:27.731390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:27.907619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab75309-e519-42f0-ae53-ddafaa73c8ed None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:27.955241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516d48b-80c5-4e36-87d8-b1a5b755caa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:27.956137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d48b-80c5-4e36-87d8-b1a5b755caa1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:27.956426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d48b-80c5-4e36-87d8-b1a5b755caa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:27.956733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d48b-80c5-4e36-87d8-b1a5b755caa1 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:27.958013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516d48b-80c5-4e36-87d8-b1a5b755caa1 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:27.959631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516d48b-80c5-4e36-87d8-b1a5b755caa1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:27.965357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 98/207] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:27 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:27.968741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:28.102118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:28.112440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6190-c48a-4bbe-a2d6-d77e84cfead0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:28.168372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:28.169077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.169360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.169631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.170471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] RBAC: Adding query filter params (role.id=4a579dca3f24408ab6efbd3f9f1e1ed6, scope.project.id=059d0856d2f94b658054f1a53c494081, user.id=dde9c5630eb64351a7888ba1a80aa0ab) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:28.170782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:28.172232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a40af-d00e-4424-8342-b1bdf6905401 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:28.181302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 110/208] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:27 2022] GET /identity/v3/role_assignments?user.id=dde9c5630eb64351a7888ba1a80aa0ab&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:28.992158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4f50-7324-4aa0-aa33-ae653eef4152 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.992704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4f50-7324-4aa0-aa33-ae653eef4152 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.992775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4f50-7324-4aa0-aa33-ae653eef4152 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.994998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 99/209] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:29.001005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9bce-b4f3-4f7b-be83-0ba97ecfbcb4 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.001302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9bce-b4f3-4f7b-be83-0ba97ecfbcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.001586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9bce-b4f3-4f7b-be83-0ba97ecfbcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.023610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bfef9bce-b4f3-4f7b-be83-0ba97ecfbcb4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:29.025551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bfef9bce-b4f3-4f7b-be83-0ba97ecfbcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:29.053367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96564-c1bd-4236-a56f-e24ecd14af66 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.053669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96564-c1bd-4236-a56f-e24ecd14af66 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.053939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da96564-c1bd-4236-a56f-e24ecd14af66 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.055444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 100/210] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:29.057498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada04b5-3548-4792-aa6d-350574254d43 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.057792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada04b5-3548-4792-aa6d-350574254d43 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.058075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada04b5-3548-4792-aa6d-350574254d43 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.059387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 101/211] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:29.065229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.065526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.065809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.086449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:29.088413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:29.100282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 111/212] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:29.101617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.102091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.102232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.122852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:29.124392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:29.169071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.174819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 102/213] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:29.176674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.213890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 112/214] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:29.214590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.232722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.328897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5b7b-13cc-4a78-a596-d630218e0ca0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.350159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c526a-0926-437e-8077-598f714ed0d1 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.380055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.381172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.381607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.382032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.396414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:29.397088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.397924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.398205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.398480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.400423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:29.401794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:29.403515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 113/215] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:29.404321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:29.406823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:29.407760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:29.408593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 103/216] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:29.410581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.481080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.505372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.524152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495ebd-c07a-4931-9557-ff7227868b04 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.528960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2936861-a279-4f44-9ef4-8365ca15b170 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.564648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.565449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.565835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.566111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.566270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.567117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.567433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.567782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.568668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:29.570227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:29.577225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:29.584872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:29.585940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:29.586721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 114/217] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:29.587981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.676398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 104/218] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:29 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:29.677640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.717683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4604-d151-4a04-a4b2-9f0effc0e3ed None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.765506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.766834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.767376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.767846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.769087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:29.769645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:29.771363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:29.783599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 115/219] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:29.792623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.797724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.818438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaad44c-b94b-4bcf-aae6-a916ae66887e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:29.839351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:29.865187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14063640-8276-475c-96d7-3982891694fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:29.865883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14063640-8276-475c-96d7-3982891694fa None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:29.866171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14063640-8276-475c-96d7-3982891694fa None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:29.866451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14063640-8276-475c-96d7-3982891694fa None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:29.867968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14063640-8276-475c-96d7-3982891694fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:29.868805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14063640-8276-475c-96d7-3982891694fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:29.871480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14063640-8276-475c-96d7-3982891694fa None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:29.888801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 105/220] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:29.895219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14063640-8276-475c-96d7-3982891694fa None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:29.947803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eb67f-fa00-42e8-962c-644fb8cfea4d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.003744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.004921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.005206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.005484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] PATH_INFO: `/v3/users/86b67f3eab2a4db6866e98f7b054d93f` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.021859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14063640-8276-475c-96d7-3982891694fa None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.022346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] RBAC: Authorizing `identity:get_user(user_id=86b67f3eab2a4db6866e98f7b054d93f)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.023807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.027445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 116/221] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/users/86b67f3eab2a4db6866e98f7b054d93f => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:30.033764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.073029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.074300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.074724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.075138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] PATH_INFO: `/v3/users/213f04f9218f43408f0e565f33093ea7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.077294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] RBAC: Authorizing `identity:get_user(user_id=213f04f9218f43408f0e565f33093ea7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.079115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.081429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 106/222] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:29 2022] GET /identity/v3/users/213f04f9218f43408f0e565f33093ea7 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:30.086906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.123498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:30.156865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:30.179092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bd975-2e02-4503-8510-83a142447bcc None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.224971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d63f5d-8abb-4712-80b7-92b2db3ae969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.225646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63f5d-8abb-4712-80b7-92b2db3ae969 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.225933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63f5d-8abb-4712-80b7-92b2db3ae969 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.226214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d63f5d-8abb-4712-80b7-92b2db3ae969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.230597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73ba98-3e1d-422c-877d-7134a6d10d7d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.277748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9906f-6eb5-40b4-81ae-b726f2017b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.278762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9906f-6eb5-40b4-81ae-b726f2017b97 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.279476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9906f-6eb5-40b4-81ae-b726f2017b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.279762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9906f-6eb5-40b4-81ae-b726f2017b97 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.281371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9906f-6eb5-40b4-81ae-b726f2017b97 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.283391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9906f-6eb5-40b4-81ae-b726f2017b97 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.288592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 107/223] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:30.290815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51e35-1fbc-41e2-ae35-f7dace939179 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.291116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51e35-1fbc-41e2-ae35-f7dace939179 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.291401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51e35-1fbc-41e2-ae35-f7dace939179 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.292917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 108/224] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:30.293505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51e35-1fbc-41e2-ae35-f7dace939179 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.309403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63f5d-8abb-4712-80b7-92b2db3ae969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.310995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d63f5d-8abb-4712-80b7-92b2db3ae969 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.376221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:30.411261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51e35-1fbc-41e2-ae35-f7dace939179 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.426311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 117/225] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:30.427875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.428221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.428526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.429185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:30.448047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:30.449796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:30.453369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.454866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.454866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.455169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/213f04f9218f43408f0e565f33093ea7/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.457459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] RBAC: Authorizing `identity:create_grant(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user_id=213f04f9218f43408f0e565f33093ea7, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.459105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.528958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 118/226] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:30.530248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.551372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 109/227] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:30 2022] PUT /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/213f04f9218f43408f0e565f33093ea7/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:30.563984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.662051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:30.668205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79143a0-5cf7-48c8-8b7d-c80095c65145 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.726378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.729379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.729932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.731579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.737150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.739383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.744133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 119/228] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:30.750472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.814733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c976ec-054b-4ae5-9b4e-05d972dd76ac None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.857577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.857784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.858014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.858298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.872990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:30.882694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:30.883624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:30.884483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:30.886973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 110/229] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:30.896545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.902260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50c037-124d-4c8f-b5f8-faaa338ffc1a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.939879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.940674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:30.940999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.941766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.942183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.943243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=86b67f3eab2a4db6866e98f7b054d93f) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:30.943770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.945532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.952524 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 120/230] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/role_assignments?user.id=86b67f3eab2a4db6866e98f7b054d93f&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:31.046752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e0f6-9dbb-4251-93b9-dcae600e0418 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:31.100016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:31.101116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:31.101654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:31.102124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:31.103411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:31.103929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:31.105626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:31.123948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 111/231] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:31.130412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:31.184046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:31.267396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22aa5c-0c15-4d59-8f9b-25899ac0b4ff None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:31.305244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:31.306659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:31.306939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:31.307249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] PATH_INFO: `/v3/users/213f04f9218f43408f0e565f33093ea7` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:31.308441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] RBAC: Authorizing `identity:get_user(user_id=213f04f9218f43408f0e565f33093ea7)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:31.309923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:31.311783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 121/232] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:31 2022] GET /identity/v3/users/213f04f9218f43408f0e565f33093ea7 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:31.317621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:31.455439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:31.463974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4175b-23a4-4c01-9af1-0c27a32a7a82 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:31.510867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5559-80dd-45a0-8748-344e181068ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:31.511753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5559-80dd-45a0-8748-344e181068ba None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:31.512037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5559-80dd-45a0-8748-344e181068ba None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:31.512317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5559-80dd-45a0-8748-344e181068ba None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:31.513588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5559-80dd-45a0-8748-344e181068ba None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:31.515050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5559-80dd-45a0-8748-344e181068ba None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:31.521111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 112/233] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:31 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:31.527732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:31.664291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:31.674813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe25c1-3e43-4c92-ad1d-bc6d7a74be9f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:31.724040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e4055-f3b2-47fe-8997-86abdd9f8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:31.725323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4055-f3b2-47fe-8997-86abdd9f8385 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:31.725807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4055-f3b2-47fe-8997-86abdd9f8385 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:31.726276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4055-f3b2-47fe-8997-86abdd9f8385 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/213f04f9218f43408f0e565f33093ea7/roles/c636e98b844543518e09f390c4850083` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:31.729203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e4055-f3b2-47fe-8997-86abdd9f8385 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user_id=213f04f9218f43408f0e565f33093ea7, role_id=c636e98b844543518e09f390c4850083)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:31.730897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e4055-f3b2-47fe-8997-86abdd9f8385 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:31.845757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 122/234] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:31 2022] PUT /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/213f04f9218f43408f0e565f33093ea7/roles/c636e98b844543518e09f390c4850083 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:32.521636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8da3c-fa75-4bce-a868-bbc2d6f34fa5 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:32.522150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8da3c-fa75-4bce-a868-bbc2d6f34fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:32.522223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8da3c-fa75-4bce-a868-bbc2d6f34fa5 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:32.523809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 113/235] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:32.529946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c577f-ca96-4c5b-be0c-4f12364c2b1d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:32.530416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c577f-ca96-4c5b-be0c-4f12364c2b1d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:32.530603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c577f-ca96-4c5b-be0c-4f12364c2b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:32.553689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-516c577f-ca96-4c5b-be0c-4f12364c2b1d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:32.554910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-516c577f-ca96-4c5b-be0c-4f12364c2b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:32.655240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 123/236] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:32.698281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8da3c-fa75-4bce-a868-bbc2d6f34fa5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:32.840096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8da3c-fa75-4bce-a868-bbc2d6f34fa5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:32.856768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb322f1-c77e-450f-8a84-1a9dcef5cbb1 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:32.857112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb322f1-c77e-450f-8a84-1a9dcef5cbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:32.857327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb322f1-c77e-450f-8a84-1a9dcef5cbb1 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:32.858888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 124/237] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:32.875425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:32.882238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:32.882784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:32.900860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:32.902024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:32.902506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:32.902979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:32.904437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:32.906238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:32.908468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 114/238] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:32.914034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:32.928544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:32.928544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:32.970433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:33.067749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 125/239] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:33.094329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488fbd1-fe76-470d-8c57-a062d3d4ae9b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:33.101174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:33.121644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:33.245986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b99e37-7bcc-40e7-809e-a4106f8aa16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:33.245986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99e37-7bcc-40e7-809e-a4106f8aa16d None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.246387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99e37-7bcc-40e7-809e-a4106f8aa16d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.246551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99e37-7bcc-40e7-809e-a4106f8aa16d None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.247290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99e37-7bcc-40e7-809e-a4106f8aa16d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:33.248820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99e37-7bcc-40e7-809e-a4106f8aa16d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:33.279920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4fe7-76ba-4a45-b9e7-7c9d039e0015 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:33.322806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba45f5-e647-4478-a483-510d8292170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:33.323972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba45f5-e647-4478-a483-510d8292170e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.324452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba45f5-e647-4478-a483-510d8292170e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.324915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba45f5-e647-4478-a483-510d8292170e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.337534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba45f5-e647-4478-a483-510d8292170e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-6cba45f5-e647-4478-a483-510d8292170e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:33.348015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:33.349084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:33.349991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:33.353412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 126/240] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:33.355102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6a243-d0ba-483c-9038-4682f0adec72 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.355614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6a243-d0ba-483c-9038-4682f0adec72 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.356064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6a243-d0ba-483c-9038-4682f0adec72 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.357660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 127/241] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:33.359305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6a243-d0ba-483c-9038-4682f0adec72 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:33.380355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 115/242] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:32 2022] POST /identity/v3/users => generated 293 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:28:33.380355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.380355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.384799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.428211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:33.428902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:33.509696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:33.587328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6a243-d0ba-483c-9038-4682f0adec72 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:33.739973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:33.740935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 116/243] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:33.743750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.744655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.745419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.747452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:33.748308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:33.751272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:33.776772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 128/244] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:33.785646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:33.799134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:33.814873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:33.923535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7b195-eee3-4bcb-96f2-eb407c958073 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:33.943150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7feb13-6432-411f-94ea-8858b1469d6a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:33.973011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4796d-71c6-4e70-aa82-e5584c44edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:33.973947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4796d-71c6-4e70-aa82-e5584c44edb6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.974613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4796d-71c6-4e70-aa82-e5584c44edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.975018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4796d-71c6-4e70-aa82-e5584c44edb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.987863 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4796d-71c6-4e70-aa82-e5584c44edb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:33.988414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51398ab8-ee51-4bd3-8e7f-91b805edec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:33.989325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398ab8-ee51-4bd3-8e7f-91b805edec98 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.989605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398ab8-ee51-4bd3-8e7f-91b805edec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.989877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398ab8-ee51-4bd3-8e7f-91b805edec98 None admin] PATH_INFO: `/v3/users/89d8c96f4a984f6f942ec290fc5842fa` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.993690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-d2e4796d-71c6-4e70-aa82-e5584c44edb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:33.998510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:33.999472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:34.000782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 117/245] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:34.002531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5e1d9-9350-41ed-9613-e5bbb703602d None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:34.002531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5e1d9-9350-41ed-9613-e5bbb703602d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:34.002868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5e1d9-9350-41ed-9613-e5bbb703602d None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:34.004981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 118/246] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:34.010260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5e1d9-9350-41ed-9613-e5bbb703602d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:34.018360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51398ab8-ee51-4bd3-8e7f-91b805edec98 None admin] RBAC: Authorizing `identity:get_user(user_id=89d8c96f4a984f6f942ec290fc5842fa)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:34.020132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51398ab8-ee51-4bd3-8e7f-91b805edec98 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:34.024126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 129/247] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:33 2022] GET /identity/v3/users/89d8c96f4a984f6f942ec290fc5842fa => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:34.027205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:34.027495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:34.027824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:34.048904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:34.050428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:34.133118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:34.166791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 130/248] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:34.169185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:34.179968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5e1d9-9350-41ed-9613-e5bbb703602d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:34.237029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:34.237729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:34.238016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:34.238295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:34.239068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:34.239375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:34.241151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:34.261559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 119/249] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:34.262418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:34.402545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0b35d-cdb0-48b3-b6c4-7cb97bd92866 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:34.456949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:34.457780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:34.458073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:34.458357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:34.469094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:34.522858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ae36f-ede8-4f8c-be5c-c2717a583e60 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:34.565881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:34.568066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:34.572148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:34.573344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:34.573820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:34.574285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:34.596382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:34.597976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:34.606854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:34.610589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:34.611396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 120/250] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:34.619263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:34.742386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109312b-326d-47a8-be38-73123dfdbb2b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:34.787620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:34.789495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:34.791354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:34.791812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] PATH_INFO: `/v3/users/213f04f9218f43408f0e565f33093ea7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:34.794500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] RBAC: Authorizing `identity:get_user(user_id=213f04f9218f43408f0e565f33093ea7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:34.795983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:34.797704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 121/251] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:34 2022] GET /identity/v3/users/213f04f9218f43408f0e565f33093ea7 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:34.798473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:34.803704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 131/252] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:34.804571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:34.857249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:34.941899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:34.985717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45cd8f-f4b0-40ed-b30c-ec3383efdb42 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.023359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.024044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.024410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.025143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.028527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:35.028527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.028527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.031930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc72de-d84e-40c9-90fd-5100e9c0f86c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.036978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 122/253] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.037876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.091859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.092583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.092838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.093117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.144654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58e271-5c55-4c7f-9a72-e6f4d148662e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.168733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.170236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.188904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.200929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.201869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.202154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.202430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.203748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.205069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.205257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.213854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 123/254] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.214718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.297133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 132/255] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:35.300392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.357190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c74f2-4db9-40f5-b1a7-3f5d9282c914 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.398222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50fcd13-3496-42a7-907a-8bae6fad6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.399145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fcd13-3496-42a7-907a-8bae6fad6700 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.400196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fcd13-3496-42a7-907a-8bae6fad6700 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.400484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50fcd13-3496-42a7-907a-8bae6fad6700 None admin] PATH_INFO: `/v3/users/86b67f3eab2a4db6866e98f7b054d93f` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.403326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fcd13-3496-42a7-907a-8bae6fad6700 None admin] RBAC: Authorizing `identity:get_user(user_id=86b67f3eab2a4db6866e98f7b054d93f)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.404886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50fcd13-3496-42a7-907a-8bae6fad6700 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.407078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 124/256] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/users/86b67f3eab2a4db6866e98f7b054d93f => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.408347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2691f-26c1-43c4-85bc-7c67072bf53b None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.408715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2691f-26c1-43c4-85bc-7c67072bf53b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.408991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2691f-26c1-43c4-85bc-7c67072bf53b None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.410235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 125/257] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:35.410926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2691f-26c1-43c4-85bc-7c67072bf53b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.424339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c546e-9e19-4191-8d58-67e45a696517 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.449798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.463224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.464676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.464676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.464900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.466538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] RBAC: Adding query filter params (role.id=4a579dca3f24408ab6efbd3f9f1e1ed6, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=89d8c96f4a984f6f942ec290fc5842fa) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:35.467250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.469429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.476053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 133/258] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/role_assignments?user.id=89d8c96f4a984f6f942ec290fc5842fa&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.477324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.493797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.538302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2691f-26c1-43c4-85bc-7c67072bf53b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.582001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849427b-a5d2-4dea-8900-62f8616f2f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.582001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849427b-a5d2-4dea-8900-62f8616f2f94 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.582001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849427b-a5d2-4dea-8900-62f8616f2f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.582001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849427b-a5d2-4dea-8900-62f8616f2f94 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.583570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849427b-a5d2-4dea-8900-62f8616f2f94 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.585079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849427b-a5d2-4dea-8900-62f8616f2f94 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.588234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 126/259] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.594939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.595624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.596232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.612236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbf315-7d2a-405c-8c30-b450b2550241 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.630817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:35.632167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:35.657639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.659141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.659627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.660090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.661843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.663887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.671735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 134/260] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.673076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.716056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 127/261] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:35.721313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.784555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.835128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d1cd1-fd76-4c3d-a34a-081f5e0d0d08 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.840717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.874613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.875373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.875658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.876648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.876907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=213f04f9218f43408f0e565f33093ea7) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:35.877224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.878829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.888523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 135/262] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/role_assignments?user.id=213f04f9218f43408f0e565f33093ea7&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:35.888523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.900457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab1d4-b1ed-4f50-9020-3e2e815300d6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.982104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.983044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.983333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.983617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/86b67f3eab2a4db6866e98f7b054d93f/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.989386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user_id=86b67f3eab2a4db6866e98f7b054d93f, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.993790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:36.070653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 128/263] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:35 2022] PUT /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/86b67f3eab2a4db6866e98f7b054d93f/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:36.095624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc03633-dd71-45b9-8cdb-6bca49da70f1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:36.124300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:36.152590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:36.153458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:36.153749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:36.154028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:36.168155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:36.176743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:36.177817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:36.180596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 136/264] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:36.187670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:36.329767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784ecfd-5515-444f-b7dd-5f15768638d4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:36.359712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:36.377146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:36.382751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:36.382751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:36.382751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:36.382751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:36.382751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:36.382751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:36.396406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 129/265] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:36.404215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:36.531493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad0b7-2c81-43b0-a468-f87af30d2b67 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:36.576646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e6fbd-4f6c-49a3-b88f-dff0977e19c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:36.577544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e6fbd-4f6c-49a3-b88f-dff0977e19c6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:36.577833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e6fbd-4f6c-49a3-b88f-dff0977e19c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:36.578112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e6fbd-4f6c-49a3-b88f-dff0977e19c6 None admin] PATH_INFO: `/v3/users/213f04f9218f43408f0e565f33093ea7` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:36.579448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e6fbd-4f6c-49a3-b88f-dff0977e19c6 None admin] RBAC: Authorizing `identity:get_user(user_id=213f04f9218f43408f0e565f33093ea7)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:36.581015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e6fbd-4f6c-49a3-b88f-dff0977e19c6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:36.583314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 137/266] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:36 2022] GET /identity/v3/users/213f04f9218f43408f0e565f33093ea7 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:36.592191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:36.807879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23dd3ab-64a5-4df8-9761-6f1281a418f9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:36.875952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:36.878239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:36.879221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:36.879888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:36.901747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23fa1-9fad-44c7-bcba-a415c21e8cd1 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:36.902268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23fa1-9fad-44c7-bcba-a415c21e8cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:36.902526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23fa1-9fad-44c7-bcba-a415c21e8cd1 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:36.908071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 138/267] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:36.909751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5da0d-3f81-442e-a0ed-32120f1883b9 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:36.910146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5da0d-3f81-442e-a0ed-32120f1883b9 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:36.910531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5da0d-3f81-442e-a0ed-32120f1883b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:36.950764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-25d5da0d-3f81-442e-a0ed-32120f1883b9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:36.951991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-25d5da0d-3f81-442e-a0ed-32120f1883b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:36.990862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.020622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.023375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.049314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.084738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 139/268] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:37.086692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5c7fe-d0e6-4181-b354-ad266982fd58 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.087179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5c7fe-d0e6-4181-b354-ad266982fd58 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.087645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5c7fe-d0e6-4181-b354-ad266982fd58 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.089755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 140/269] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:37.101158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.101795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.102316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.132952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:37.135291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.202441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 130/270] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:37.203329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.220558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 141/271] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 03:28:37.221863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.334464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.342107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0c0c1-b094-4ff4-b2df-2ce2d9f64b67 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.351402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.354399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319051be-cbf1-4b56-b689-353dbecbb6ce None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.385123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.386444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.386928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.387394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.389195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.391162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.393509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 142/272] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:37.394676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.401110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.402085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.402273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.402525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.404358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.405737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.407895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 131/273] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:37.413210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.523171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d4464-b233-4c9d-9801-75a3ce91a413 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.525530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1685b-81c4-4cc0-8b90-9cc4667b9e85 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.562428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.563110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.563396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.563692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.564811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] RBAC: Adding query filter params (role.id=c636e98b844543518e09f390c4850083, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=213f04f9218f43408f0e565f33093ea7) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:37.565135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.566369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.566809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.567685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.568183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.568704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.576574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 132/274] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/role_assignments?user.id=213f04f9218f43408f0e565f33093ea7&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=c636e98b844543518e09f390c4850083 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:37.578103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.595910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:37.604722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:37.605794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:37.606584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:37.609177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 143/275] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:37.618300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.619631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.687345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.750290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd692bc-7d0e-4dbc-8c1d-d71496e89ab2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.794766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.795865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.796378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.796894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.797849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.799559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.810937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76c516-e5fa-4a83-b4aa-bcad8195f2ac None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.869824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.870861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.871407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.871931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.873002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:37.873676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.881581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 133/276] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:28:37 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:37.884079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.908012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 144/277] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:37.915984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:38.020328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:38.050336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34448c-e6a4-4c24-a9bf-8752e305e26b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:38.092219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a535e5-b4ff-41f3-8793-462bfcb184d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:38.093234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a535e5-b4ff-41f3-8793-462bfcb184d2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:38.093729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a535e5-b4ff-41f3-8793-462bfcb184d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:38.096423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a535e5-b4ff-41f3-8793-462bfcb184d2 None admin] PATH_INFO: `/v3/users/5c2967e00c9147d480547a0d991254f1` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:38.112906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a535e5-b4ff-41f3-8793-462bfcb184d2 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2967e00c9147d480547a0d991254f1)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:38.114569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a535e5-b4ff-41f3-8793-462bfcb184d2 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:38.120746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 134/278] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/users/5c2967e00c9147d480547a0d991254f1 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:38.124121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:38.231632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:38.238625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1c83e-1055-4e3c-b0b5-ff392677aaa5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:38.280578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:38.281265 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:38.281550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:38.281840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:38.282751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, user.id=5c2967e00c9147d480547a0d991254f1, scope.system=all) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:38.283072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:38.284671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b6c7c-9e06-4322-bfba-84e2258135ea None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:38.293903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 145/279] 10.222.0.58 () {56 vars in 1308 bytes} [Fri May 27 03:28:38 2022] GET /identity/v3/role_assignments?user.id=5c2967e00c9147d480547a0d991254f1&scope.system=all&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:38.640330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a269d9-bceb-4ca7-8d17-c8f446e632c7 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:38.640744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a269d9-bceb-4ca7-8d17-c8f446e632c7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:38.640992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a269d9-bceb-4ca7-8d17-c8f446e632c7 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:38.642806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 135/280] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:38.648076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:38.648368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:38.648673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:38.673673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:38.674914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:38.783256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 146/281] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:38.854426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a269d9-bceb-4ca7-8d17-c8f446e632c7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:39.015704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a269d9-bceb-4ca7-8d17-c8f446e632c7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:39.050589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:39.057329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:39.058576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.059090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.059572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.072453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:39.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:39.081917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:39.082690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:39.085170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 136/282] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:39.094735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:39.302973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:39.314122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f332c9-99cf-45e4-9e40-d1b73fe73c74 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:39.373349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:39.374052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.374333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.374603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.375367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:39.375673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:39.377168 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:39.389308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 147/283] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:39.397330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:39.548701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ba04-1e29-4430-84cc-3a05d6054590 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:39.590459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3c29a-9b3c-4efe-b9cf-97c883722118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:39.591688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3c29a-9b3c-4efe-b9cf-97c883722118 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.592163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3c29a-9b3c-4efe-b9cf-97c883722118 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.592659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3c29a-9b3c-4efe-b9cf-97c883722118 None admin] PATH_INFO: `/v3/users/89d8c96f4a984f6f942ec290fc5842fa` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.594213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3c29a-9b3c-4efe-b9cf-97c883722118 None admin] RBAC: Authorizing `identity:get_user(user_id=89d8c96f4a984f6f942ec290fc5842fa)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:39.595943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3c29a-9b3c-4efe-b9cf-97c883722118 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:39.598305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 137/284] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:39 2022] GET /identity/v3/users/89d8c96f4a984f6f942ec290fc5842fa => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:39.604369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:39.622171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9903-d90f-45a2-b833-70f4a73de941 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.622751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9903-d90f-45a2-b833-70f4a73de941 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.623208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a9903-d90f-45a2-b833-70f4a73de941 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.624879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 138/285] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:39.631234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.631817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.632302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.659021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:39.660572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:39.753291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:39.777685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 139/286] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:39.788459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c4fa-354f-49b2-a288-ff36e555e52e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:39.845711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:39.859864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:39.860771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.861056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.861339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.863340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:39.866927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:39.871464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 148/287] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:39 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:39.880017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:39.932331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:40.046027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e12aee-b3d6-42ce-a07d-e0b44be0e1d8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:40.072060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ccf5d-0421-42ac-92e0-9f61b78cdeb7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:40.108548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:40.110624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:40.111580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:40.112909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/89d8c96f4a984f6f942ec290fc5842fa/roles/4a579dca3f24408ab6efbd3f9f1e1ed6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:40.120523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user_id=89d8c96f4a984f6f942ec290fc5842fa, role_id=4a579dca3f24408ab6efbd3f9f1e1ed6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:40.120523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:40.121390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:40.122304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:40.122593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:40.122924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:40.134419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:40.142375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:40.143339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:40.144124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 140/288] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:40.151591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:40.203595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:40.207812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 149/289] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:28:39 2022] PUT /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/users/89d8c96f4a984f6f942ec290fc5842fa/roles/4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:40.284936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3bd72-b32b-4dd8-85b1-927a1fb16d40 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:40.377838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:40.379029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:40.379450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:40.379851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:40.381066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:40.381523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:40.384471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:40.400855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 141/290] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:40.406070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:40.545105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:40.565082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9d1d1-d847-4cb5-84b9-c3edec5f73d6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:40.613203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:40.614562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:40.615542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:40.615842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] PATH_INFO: `/v3/users/86b67f3eab2a4db6866e98f7b054d93f` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:40.617261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] RBAC: Authorizing `identity:get_user(user_id=86b67f3eab2a4db6866e98f7b054d93f)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:40.619201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:40.621287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 150/291] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:40 2022] GET /identity/v3/users/86b67f3eab2a4db6866e98f7b054d93f => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:40.627063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:40.799687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f711e-c165-4750-b329-9ef12b8509b3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:40.818577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:40.849416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f300b-a8de-4343-8437-dbad494a3814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:40.850080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f300b-a8de-4343-8437-dbad494a3814 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:40.850397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f300b-a8de-4343-8437-dbad494a3814 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:40.850673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f300b-a8de-4343-8437-dbad494a3814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:40.975383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f300b-a8de-4343-8437-dbad494a3814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:40.980285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f300b-a8de-4343-8437-dbad494a3814 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:41.134262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 142/292] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:41.139369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:41.215846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442266-e8ed-4bb1-a81d-62dcf104e610 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.216150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442266-e8ed-4bb1-a81d-62dcf104e610 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.216406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442266-e8ed-4bb1-a81d-62dcf104e610 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.218253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 143/293] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:41.223540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313c1ad-6ce8-4932-b2e8-229c871ac102 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.223838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313c1ad-6ce8-4932-b2e8-229c871ac102 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.224491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313c1ad-6ce8-4932-b2e8-229c871ac102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.254152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4313c1ad-6ce8-4932-b2e8-229c871ac102 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:41.255359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4313c1ad-6ce8-4932-b2e8-229c871ac102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:41.266916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:41.269271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2a356-6841-4b53-a11b-cce7b8e158bd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:41.312822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b00b9-4b0b-42b1-9c93-1f5c2cc9c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:41.313714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b00b9-4b0b-42b1-9c93-1f5c2cc9c591 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.314001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b00b9-4b0b-42b1-9c93-1f5c2cc9c591 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.314282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b00b9-4b0b-42b1-9c93-1f5c2cc9c591 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.315566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b00b9-4b0b-42b1-9c93-1f5c2cc9c591 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:41.317093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b00b9-4b0b-42b1-9c93-1f5c2cc9c591 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:41.319262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 151/294] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:41 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:41.324957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82446c6-597e-4eda-a477-bd543e98afad None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.325316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82446c6-597e-4eda-a477-bd543e98afad None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.325571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82446c6-597e-4eda-a477-bd543e98afad None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.331558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 152/295] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:41.331558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82446c6-597e-4eda-a477-bd543e98afad None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:41.353199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:41.365791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 144/296] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:41.368411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.369099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.369560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.391743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:41.393278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:41.470471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82446c6-597e-4eda-a477-bd543e98afad None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:41.507092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 145/297] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:41.508999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:41.544636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:41.545914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.546486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.546924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.548696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=86b67f3eab2a4db6866e98f7b054d93f) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:41.549338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:41.552213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:41.561493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 153/298] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:41 2022] GET /identity/v3/role_assignments?user.id=86b67f3eab2a4db6866e98f7b054d93f&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:41.562860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:41.619501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:41.664460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e1c5b-06f8-43c9-884c-93bf1d4c185c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:41.682052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:41.722012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be6801-cb21-4236-b2a6-9788769daf86 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:41.735558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948474c8-bb42-4243-b3f8-0f0508623123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:41.737037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948474c8-bb42-4243-b3f8-0f0508623123 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.737528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948474c8-bb42-4243-b3f8-0f0508623123 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.737983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948474c8-bb42-4243-b3f8-0f0508623123 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.767137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948474c8-bb42-4243-b3f8-0f0508623123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:41.774538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa06775-847f-4e16-b7ff-c1b5ba01e7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:41.775716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa06775-847f-4e16-b7ff-c1b5ba01e7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.776207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa06775-847f-4e16-b7ff-c1b5ba01e7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.776764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa06775-847f-4e16-b7ff-c1b5ba01e7f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.778513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa06775-847f-4e16-b7ff-c1b5ba01e7f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:41.780326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa06775-847f-4e16-b7ff-c1b5ba01e7f0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:41.782807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 154/299] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-948474c8-bb42-4243-b3f8-0f0508623123 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:41.783112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:41.783965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:41.784797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:41.787584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9b842-4bcf-4e93-9e16-32ca0632bd4b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.788147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9b842-4bcf-4e93-9e16-32ca0632bd4b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.788659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9b842-4bcf-4e93-9e16-32ca0632bd4b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.791568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 155/300] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:41.795391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 146/301] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:41.796785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9b842-4bcf-4e93-9e16-32ca0632bd4b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:41.798424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:41.798719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:41.799000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:41.822561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:41.824381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:41.928117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 147/302] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:41.929031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:41.959125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9b842-4bcf-4e93-9e16-32ca0632bd4b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:41.999596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.000272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.000700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.001640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] PATH_INFO: `/v3/groups` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.001752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.004006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.004868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:42.049035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33951-6f92-466e-bb35-7244a7c76e36 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.081417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 156/303] 10.222.0.58 () {60 vars in 1134 bytes} [Fri May 27 03:28:41 2022] POST /identity/v3/groups => generated 229 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:42.085742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.087984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.088361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.089117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:42.089433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.090972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.093584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.110674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:42.111314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 148/304] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:42.120540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.238894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cac0f-94ec-4655-8602-14d1e0b39755 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.284229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.285082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0c06-b983-4a1d-9a54-d407a9cd83a6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.285780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.285780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.285780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.299637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:42.310523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:42.312274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:42.314025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 157/305] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:42.315296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:42.321472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.326647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.327812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.328037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.328235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] PATH_INFO: `/v3/users/687867f558f248d89aa9d5a771b04a47` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.347320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] RBAC: Authorizing `identity:get_user(user_id=687867f558f248d89aa9d5a771b04a47)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.356300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.356300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 149/306] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:42 2022] GET /identity/v3/users/687867f558f248d89aa9d5a771b04a47 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:42.356542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.418639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:42.454911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6defa3-ed13-4cb9-affa-eb37cecdcb10 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.495289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.495888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.496177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.496457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.497258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:42.497571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.498255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323f87-f466-48af-b067-aa10291b1f98 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.498996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.513815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 158/307] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:42.514973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.552045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.553457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.555506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.556777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.557896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] RBAC: Adding query filter params (role.id=4a579dca3f24408ab6efbd3f9f1e1ed6, user.id=687867f558f248d89aa9d5a771b04a47, scope.system=all) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:42.558424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.560927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.572289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 150/308] 10.222.0.58 () {56 vars in 1308 bytes} [Fri May 27 03:28:42 2022] GET /identity/v3/role_assignments?user.id=687867f558f248d89aa9d5a771b04a47&scope.system=all&role.id=4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:42.653803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d5b6e-a5e3-4c4c-a652-fd671227c16f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.698914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:42.709795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107570c-43df-4161-b264-39822214b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.712210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107570c-43df-4161-b264-39822214b15b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.713512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107570c-43df-4161-b264-39822214b15b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.714313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107570c-43df-4161-b264-39822214b15b None admin] PATH_INFO: `/v3/users/5c2967e00c9147d480547a0d991254f1` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.717212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107570c-43df-4161-b264-39822214b15b None admin] RBAC: Authorizing `identity:get_user(user_id=5c2967e00c9147d480547a0d991254f1)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.720615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107570c-43df-4161-b264-39822214b15b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.724271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 159/309] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:42 2022] GET /identity/v3/users/5c2967e00c9147d480547a0d991254f1 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:42.732550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.800832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:42.864796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f726c-39e1-4158-8a39-af7e90fc7667 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.905149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff17d-964f-49f9-b5ce-6bbca3973fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.906046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff17d-964f-49f9-b5ce-6bbca3973fb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.906331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff17d-964f-49f9-b5ce-6bbca3973fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.906610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff17d-964f-49f9-b5ce-6bbca3973fb9 None admin] PATH_INFO: `/v3/system/users/5c2967e00c9147d480547a0d991254f1/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.908326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeff17d-964f-49f9-b5ce-6bbca3973fb9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5c2967e00c9147d480547a0d991254f1, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.909792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeff17d-964f-49f9-b5ce-6bbca3973fb9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.999877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 151/310] 10.222.0.58 () {58 vars in 1334 bytes} [Fri May 27 03:28:42 2022] PUT /identity/v3/system/users/5c2967e00c9147d480547a0d991254f1/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:43.233940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf950942-bea0-4109-bf55-501ef47f1e8b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:43.234196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf950942-bea0-4109-bf55-501ef47f1e8b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:43.234399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf950942-bea0-4109-bf55-501ef47f1e8b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:43.236103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 160/311] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:43.246625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:43.247559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:43.248334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:43.278586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:43.281244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:43.387046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 152/312] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:43.417155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf950942-bea0-4109-bf55-501ef47f1e8b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:43.535663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf950942-bea0-4109-bf55-501ef47f1e8b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:43.575115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:43.575924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:43.576201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:43.576474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:43.588590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:43.596301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:43.598088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:43.603315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 161/313] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:43.608477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:43.653903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:43.725563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5dc5-4d9e-4f73-9391-e1d1ba76cfa8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:43.772841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:43.774391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:43.774874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:43.775372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:43.776426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:43.777009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:43.778658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:43.795230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 153/314] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:43.803177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:43.877645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:43.958303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3cb92-4412-4b45-ab73-a11a8e037f8d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:44.018313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:44.019841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.020340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.020833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] PATH_INFO: `/v3/users/89d8c96f4a984f6f942ec290fc5842fa` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.023051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] RBAC: Authorizing `identity:get_user(user_id=89d8c96f4a984f6f942ec290fc5842fa)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:44.025780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:44.029097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 162/315] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:43 2022] GET /identity/v3/users/89d8c96f4a984f6f942ec290fc5842fa => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:44.034732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:44.151678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae405-fa6d-42c8-b454-30d8124ee3e6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:44.153336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:44.191602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:44.192280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.192597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.192881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.271813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:44.273339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:44.392686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 154/316] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:44.394999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:44.549223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81714b12-b7e5-4fb3-a727-69f4b7d54e4a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:44.582983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:44.596177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65ab6f-63e5-42c9-802b-997a515c0aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:44.597491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ab6f-63e5-42c9-802b-997a515c0aae None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.597956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ab6f-63e5-42c9-802b-997a515c0aae None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.598474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65ab6f-63e5-42c9-802b-997a515c0aae None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.600701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65ab6f-63e5-42c9-802b-997a515c0aae None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:44.602820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65ab6f-63e5-42c9-802b-997a515c0aae None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:44.605519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 163/317] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:44 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:44.616541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:44.626433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:44.628490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48969f1-20b4-4b01-9945-26137fcfcfd9 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.629084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48969f1-20b4-4b01-9945-26137fcfcfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.629354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48969f1-20b4-4b01-9945-26137fcfcfd9 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.630816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 164/318] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:44.637482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.637849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.643040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.661597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:44.662834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:44.744650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05b353-0d4c-4ba0-ae82-807a8d8109d5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:44.748978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 165/319] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:44.772556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:44.795211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:44.795994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.796384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.796975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.798567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] RBAC: Adding query filter params (role.id=4a579dca3f24408ab6efbd3f9f1e1ed6, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d, user.id=89d8c96f4a984f6f942ec290fc5842fa) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:44.799202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:44.801518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:44.815667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 155/320] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:28:44 2022] GET /identity/v3/role_assignments?user.id=89d8c96f4a984f6f942ec290fc5842fa&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:44.913882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdeb113-3fc7-4265-aefc-132728abd2e2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:44.946596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:44.959078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f765d-882f-4088-b944-b13076d233d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:44.960287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f765d-882f-4088-b944-b13076d233d1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:44.960803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f765d-882f-4088-b944-b13076d233d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:44.961967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f765d-882f-4088-b944-b13076d233d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:44.963637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f765d-882f-4088-b944-b13076d233d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:44.965590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f765d-882f-4088-b944-b13076d233d1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:44.967991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 166/321] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:44.976277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.080837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:45.130742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a23656-5007-472f-9fc1-ba6829e38d46 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.131365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a23656-5007-472f-9fc1-ba6829e38d46 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.131847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a23656-5007-472f-9fc1-ba6829e38d46 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.132186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849870f-c596-468f-80a1-10e299ae478d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:45.133947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 167/322] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:45.139903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.140584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.141003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.169748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:45.169748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:45.191927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:45.192902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.193341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.193770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] PATH_INFO: `/v3/groups` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.194658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:45.198160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:45.252319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 168/323] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:45.275584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.285400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 156/324] 10.222.0.58 () {60 vars in 1134 bytes} [Fri May 27 03:28:44 2022] POST /identity/v3/groups => generated 226 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:45.403866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7079-f8b3-46ec-ac93-66569270a4e8 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:45.430233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:45.449760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:45.450945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.450945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.451216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.465232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:45.473201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:45.474200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:45.480800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 169/325] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:45.482662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.608554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d1796-e74a-43d1-b23a-b56d96a11a31 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:45.638440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:45.647210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:45.647890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.648180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.648461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.649906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:45.650217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:45.651609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f98885-3920-46bb-aeba-a5f775564ad9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:45.665477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 157/326] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:45.671329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.684452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ee3aa-6c8a-403f-84b9-fb732d5b03e5 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.684777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ee3aa-6c8a-403f-84b9-fb732d5b03e5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.685029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ee3aa-6c8a-403f-84b9-fb732d5b03e5 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.686654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 158/327] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:45.693310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.694134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.694134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.715050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:45.716803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:45.800877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 159/328] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:45.804543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befdf88-e1f8-4443-add3-5f399ed7a8aa None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:45.826745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.846455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:45.847566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.848215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.848490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] PATH_INFO: `/v3/users/687867f558f248d89aa9d5a771b04a47` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.849802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] RBAC: Authorizing `identity:get_user(user_id=687867f558f248d89aa9d5a771b04a47)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:45.851871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:45.854370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 170/329] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:45 2022] GET /identity/v3/users/687867f558f248d89aa9d5a771b04a47 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:45.859923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.949819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:45.979487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cff0b-1963-4322-93da-85c4f22aa31c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:46.001075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:46.026762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:46.027577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:46.027987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:46.028272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:46.042024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:46.045435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5bc02-7e4f-469c-9617-3e23575acb99 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:46.052106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:46.053073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:46.053861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 160/330] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:46.061513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:46.115785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:46.116748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:46.117034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:46.117925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] PATH_INFO: `/v3/system/users/687867f558f248d89aa9d5a771b04a47/roles/4a579dca3f24408ab6efbd3f9f1e1ed6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:46.119935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=687867f558f248d89aa9d5a771b04a47, role_id=4a579dca3f24408ab6efbd3f9f1e1ed6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:46.121520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:46.184382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657e829-ebe1-44dd-aea1-852cd11e558c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:46.190204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 171/331] 10.222.0.58 () {58 vars in 1334 bytes} [Fri May 27 03:28:45 2022] PUT /identity/v3/system/users/687867f558f248d89aa9d5a771b04a47/roles/4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:28:46.234600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:46.235272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:46.235565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:46.235845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:46.236726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:46.237054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:46.238593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:46.249604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 161/332] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:46.255709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:46.407398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f632b9-685d-49b3-9c45-ebc0424e8e84 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:46.425602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:46.453327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481988a-33fe-4b1e-a92e-f12b236dde29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:46.454231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481988a-33fe-4b1e-a92e-f12b236dde29 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:46.454513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481988a-33fe-4b1e-a92e-f12b236dde29 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:46.454934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481988a-33fe-4b1e-a92e-f12b236dde29 None admin] PATH_INFO: `/v3/users/5c2967e00c9147d480547a0d991254f1` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:46.461548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481988a-33fe-4b1e-a92e-f12b236dde29 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2967e00c9147d480547a0d991254f1)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:46.463083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481988a-33fe-4b1e-a92e-f12b236dde29 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:46.465502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 172/333] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:46 2022] GET /identity/v3/users/5c2967e00c9147d480547a0d991254f1 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:46.471433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:46.544074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:46.619398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f602186-d95c-4691-a3e9-664318d3f468 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:46.660706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:46.661418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:46.661734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:46.662096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:46.662937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] RBAC: Adding query filter params (role.id=93828c8fe28c41ca831b85d55b1a50ea, user.id=5c2967e00c9147d480547a0d991254f1, scope.system=all) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:46.663249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:46.665193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22b4ea-da40-4d82-98a0-ecb92538a002 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:46.679819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 162/334] 10.222.0.58 () {56 vars in 1308 bytes} [Fri May 27 03:28:46 2022] GET /identity/v3/role_assignments?user.id=5c2967e00c9147d480547a0d991254f1&scope.system=all&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:48.232214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee393-90b0-4788-9c39-ba2c3bf980f3 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:48.233014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee393-90b0-4788-9c39-ba2c3bf980f3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:48.233014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ee393-90b0-4788-9c39-ba2c3bf980f3 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:48.234505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 173/335] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:48.240437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:48.241749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:48.242914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:48.264430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:48.268153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:48.413217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 163/336] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:48.478354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ee393-90b0-4788-9c39-ba2c3bf980f3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:48.600109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ee393-90b0-4788-9c39-ba2c3bf980f3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:48.647548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:48.649054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:48.649570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:48.650040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:48.666727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:48.674695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:48.675590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:48.676386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:48.679106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 174/337] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:48.685222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:48.720094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:48.812611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba59269-7f12-44e8-937c-9ad889a971d3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:48.857426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:48.859120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:48.859935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:48.860789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:48.862577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:48.863466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:48.866464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b35b5-5a1d-4a74-9295-0896a1299cd1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:48.891155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 164/338] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:48.897948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:48.998717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:49.116428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2194-cad9-4e23-a477-e3ee030cbca6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:49.193663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:49.194708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.195155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.195697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.271711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f2889-ecae-4fd5-a766-b74cbe0c2a53 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.272710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f2889-ecae-4fd5-a766-b74cbe0c2a53 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.273191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f2889-ecae-4fd5-a766-b74cbe0c2a53 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.279671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 165/339] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:49.291329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2098db-b05b-42ba-b72a-cfd9f5354a63 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.292150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2098db-b05b-42ba-b72a-cfd9f5354a63 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.292911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2098db-b05b-42ba-b72a-cfd9f5354a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.313140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:49.314648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:49.333645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4e2098db-b05b-42ba-b72a-cfd9f5354a63 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:49.334847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4e2098db-b05b-42ba-b72a-cfd9f5354a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:49.402433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:49.417911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 166/340] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:49.419268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faee820-d129-4197-a373-7c355baf429b None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.419557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faee820-d129-4197-a373-7c355baf429b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.419833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faee820-d129-4197-a373-7c355baf429b None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.422794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 167/341] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:49.430633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.431388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.432043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.467322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:49.470339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:49.497440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:49.544147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 175/342] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:49.545300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:49.566366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 168/343] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:49.567522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:49.691947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81292a-0068-4adc-aec5-b5456ee7385e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:49.692252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6176c-3f37-4da5-a4b5-6f3a511430d9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:49.743110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:49.744461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.744777 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.745151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.751136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:49.752309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.752785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.753215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.754907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:49.756607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:49.759000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 169/344] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:49 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:49.760058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:49.764799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:49.781258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:49.782227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:49.783021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 176/345] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:49.784196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:49.790053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:49.793147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:49.884788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3733bc0-3d56-4fa0-8a7b-4629bd5bce22 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:49.937417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:49.938586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:49.939066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:49.939563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:49.947244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c44c8-cbfa-467c-b14a-436292d40577 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:49.953079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:49.962125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:49.963892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:49.965345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:49.969986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 170/346] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:49.971480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.014254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.015443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.016028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.016594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.032852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.036048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.040153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 177/347] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:49 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:50.046769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.106404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.135954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.138584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069672ae-5ee7-4ed1-9ccd-f2ede345d1ed None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:50.178090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.179071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.179554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.180469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.181420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:50.181874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.182256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1050b-9c3a-44f8-8761-6426adbddb0f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:50.183642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.194341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 171/348] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:50.195481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.230591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.231363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.231866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.232339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.233344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:50.233948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.235675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6296af4-0607-4706-af3d-ec4f83dfbf28 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.247473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 178/349] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:50.249338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9cfbd-ea62-44c8-a373-a4b19764197e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.249852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9cfbd-ea62-44c8-a373-a4b19764197e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.250445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9cfbd-ea62-44c8-a373-a4b19764197e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.252875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 179/350] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:50.254260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9cfbd-ea62-44c8-a373-a4b19764197e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.307031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994455fb-c412-42e0-a978-829b51f7ff1e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:50.347050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.355480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.355480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.355480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.355480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=059d0856d2f94b658054f1a53c494081) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:50.355480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.355480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.357769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 172/351] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:50.358987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.363361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.388705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9cfbd-ea62-44c8-a373-a4b19764197e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:50.431818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.442585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f7016-30a4-4b67-a631-0d61bf7e441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.443780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f7016-30a4-4b67-a631-0d61bf7e441a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.444284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f7016-30a4-4b67-a631-0d61bf7e441a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.444868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f7016-30a4-4b67-a631-0d61bf7e441a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.487784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf329011-c40d-4072-bcbe-4647155ab617 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:50.528052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.529055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.529055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.529385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.531830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f7016-30a4-4b67-a631-0d61bf7e441a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.533993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f7016-30a4-4b67-a631-0d61bf7e441a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.610418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.611722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.615020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.661094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 180/352] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:50.663110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.663670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.664275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.684468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:50.686349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:50.720867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.726138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 173/353] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:28:50.727057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.798816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 181/354] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:50.801157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.846775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11480d02-bd33-488d-aa15-c2a458b5daf0 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:50.849208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:50.902686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:50.903441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:50.904051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:50.904539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:50.915749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:50.915749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:50.915749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 174/355] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:50.915749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:50.969636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70efc3-1be6-4b05-bceb-d4a1c89befea None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.011257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.012490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.013124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.013600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.015074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:51.016939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:51.019371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 182/356] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:51.020439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.051651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:51.080000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee14ddb-7d37-4db9-81af-d8383056d692 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.127259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.128583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.129007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.129418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.142004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:51.146450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cebbc4-c175-4bdb-9b87-7493a1e97a33 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:51.150699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:51.151610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:51.152439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 175/357] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:51.153570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.210098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6baaf-fdf6-4b61-be3e-f3e082da96d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.213239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6baaf-fdf6-4b61-be3e-f3e082da96d7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.213659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6baaf-fdf6-4b61-be3e-f3e082da96d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.214044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6baaf-fdf6-4b61-be3e-f3e082da96d7 None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.216048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6baaf-fdf6-4b61-be3e-f3e082da96d7 None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:51.218724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6baaf-fdf6-4b61-be3e-f3e082da96d7 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:51.221823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 183/358] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:51.222018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:51.223890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ff38-3350-4326-b7b6-0eefffe195fc None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.224378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ff38-3350-4326-b7b6-0eefffe195fc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.224882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4ff38-3350-4326-b7b6-0eefffe195fc None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.227792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 184/359] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:51.229793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eeb78-d3ae-4e0a-ba74-3c5fe6936e0b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.230267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eeb78-d3ae-4e0a-ba74-3c5fe6936e0b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.230648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493eeb78-d3ae-4e0a-ba74-3c5fe6936e0b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.233228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 185/360] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:51.234586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eeb78-d3ae-4e0a-ba74-3c5fe6936e0b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.342386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142504e1-814f-49fe-b14e-2d1868a45bc5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.397828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.400974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.401306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.401569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.404031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:51.405645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:51.407660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 176/361] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:51.408179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.411495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:51.421759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493eeb78-d3ae-4e0a-ba74-3c5fe6936e0b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.478948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.479978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.480405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.480865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.481839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:51.482298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:51.483914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de06ee-c89e-40b9-92a5-0c266aa20a71 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:51.496271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 186/362] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:51.498213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f194-abc6-4fbe-987d-2e7da6a1e4f6 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.498660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f194-abc6-4fbe-987d-2e7da6a1e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.499072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f194-abc6-4fbe-987d-2e7da6a1e4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.533681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3299f194-abc6-4fbe-987d-2e7da6a1e4f6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:51.535126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3299f194-abc6-4fbe-987d-2e7da6a1e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:51.569578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:51.575297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a854d04-243c-440c-9639-8e8586531af4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.632423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 187/363] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:51.633410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.633673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.633897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.634179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.635029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=c636e98b844543518e09f390c4850083, scope.project.id=059d0856d2f94b658054f1a53c494081) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:51.635346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:51.636065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.636588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.637047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:51.637254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.645616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 177/364] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=c636e98b844543518e09f390c4850083 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:51.646550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.657436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:51.659023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:51.764810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 188/365] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:51.766224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.799744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12f0cc-6ef0-421f-9780-48253f8501af None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:51.802770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:51.849562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:51.851202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:51.852285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:51.853004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:51.854608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:51.855120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:51.857202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:51.874880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 178/366] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:51.875923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:51.936945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:51.944998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2073b79-497a-4104-80e2-ec8776420511 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.005607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.007093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.009699 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] PATH_INFO: `/v3/users/687867f558f248d89aa9d5a771b04a47` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.016277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] RBAC: Authorizing `identity:get_user(user_id=687867f558f248d89aa9d5a771b04a47)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.017840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.019645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 189/367] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/users/687867f558f248d89aa9d5a771b04a47 => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:52.020418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.095982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c42b-360f-462e-a65f-12e1b902298d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.169741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.171124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.171618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.173692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.185901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:52.207048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:52.209547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32396d-3e30-4dc4-9c48-5d92c1062c22 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:52.219894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:52.221796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:52.223598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:52.229956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 179/368] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:52.236539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.278326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.279184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.279467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.279740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.296920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:52.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:52.308527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:52.310087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 190/369] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:52.311283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.465378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:52.493959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23feb2dd-dde3-4119-b99e-8afbfc0d1ada None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.506760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5e531-b84e-4f44-acb7-98e471a04235 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.550459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.551131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.551420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.551702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.552724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] RBAC: Adding query filter params (role.id=4a579dca3f24408ab6efbd3f9f1e1ed6, user.id=687867f558f248d89aa9d5a771b04a47, scope.system=all) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:52.553047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.554636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.575939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 180/370] 10.222.0.58 () {56 vars in 1308 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/role_assignments?user.id=687867f558f248d89aa9d5a771b04a47&scope.system=all&role.id=4a579dca3f24408ab6efbd3f9f1e1ed6 => generated 510 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:52.577845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.607395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.608596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.609104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.609580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.610928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:52.611488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.614562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.644698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 191/371] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:52.654815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.669453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:52.833238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e78a4-cb06-47ee-86d0-0aa38c3db0fe None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.868045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a2d06-b902-4ad4-948c-1f1960210051 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.882294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.886816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.887331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.887818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.888940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:52.889476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.891299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.912924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 181/372] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:52.921022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.942540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:52.972397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.973114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.973405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.973679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:53.171969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:53.174189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:53.188291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:53.225236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ccbca-0c96-4ba6-ab61-5463bcc3201c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:53.301147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:53.301828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:53.302116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:53.302394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:53.408345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 192/373] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:53.422870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:53.437291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:53.439102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:53.461110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:53.661947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d5771-8487-48a8-bb97-100158715e25 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:53.674016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:53.701669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 182/374] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 783 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:28:53.707653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:53.708679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:53.709005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:53.709250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:53.709398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:53.710650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:53.713317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:53.715383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 193/375] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:53 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:53.726682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:53.857020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b68b1-6943-4386-889b-49f89dd5fb08 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:53.907780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:53.909660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:53.910169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:53.912804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:53.921157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:53.924890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:53.932644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 194/376] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:53 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:53.935991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:53.983289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d324f70-2ede-4737-95e1-0487358b4a12 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:53.998129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:54.027109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:54.071600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:54.071600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.071600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.071600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] PATH_INFO: `/v3/projects/1781b8350b784207b421bac2bc80c855` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.077801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] RBAC: Authorizing `identity:get_project(project_id=1781b8350b784207b421bac2bc80c855)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:54.080443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:54.088667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 183/377] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:53 2022] GET /identity/v3/projects/1781b8350b784207b421bac2bc80c855 => generated 344 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:54.093703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:54.101463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47104bbe-51e0-49d0-86bf-ef4e34bdd19f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:54.142695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:54.143748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.144242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.144766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.145823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=c636e98b844543518e09f390c4850083, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:54.146365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:54.148102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b351c-8779-4685-84d3-6a3047910de1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:54.155532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 195/378] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:28:53 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=c636e98b844543518e09f390c4850083 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:54.275198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3088-99d9-4b4e-82cf-cf124e5e2951 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.275877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3088-99d9-4b4e-82cf-cf124e5e2951 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.276402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3088-99d9-4b4e-82cf-cf124e5e2951 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.278256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 196/379] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:54.286831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.287251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.287678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.293043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a86208-d609-418b-a07c-2f081be0cd62 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:54.336563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:54.336962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:54.374345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:54.375925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.377023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.377352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] PATH_INFO: `/v3/groups/edda4c9ce90a432e92a2b2b21a0965a7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.401551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] RBAC: Authorizing `identity:get_group(group_id=edda4c9ce90a432e92a2b2b21a0965a7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:54.404514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:54.407852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 184/380] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:54 2022] GET /identity/v3/groups/edda4c9ce90a432e92a2b2b21a0965a7 => generated 229 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:54.420965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:54.524024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 197/381] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:54.561826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:54.580243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:54.657965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:54.671239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11557d6-1885-4ee2-b42c-778b3bef5643 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:54.754866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b92e-6edc-48ff-abdb-56bc8d400925 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:54.757006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:54.757925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.758289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.758571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.759431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] RBAC: Adding query filter params (group.id=edda4c9ce90a432e92a2b2b21a0965a7, role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=1781b8350b784207b421bac2bc80c855) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:54.759747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:54.761283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:54.776195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 185/382] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:28:54 2022] GET /identity/v3/role_assignments?group.id=edda4c9ce90a432e92a2b2b21a0965a7&scope.project.id=1781b8350b784207b421bac2bc80c855&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 260 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:54.813307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04cdb33-644b-4fcc-af99-77bce92c6e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:54.814076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cdb33-644b-4fcc-af99-77bce92c6e20 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:54.814361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cdb33-644b-4fcc-af99-77bce92c6e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:54.814637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04cdb33-644b-4fcc-af99-77bce92c6e20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:54.837741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04cdb33-644b-4fcc-af99-77bce92c6e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f04cdb33-644b-4fcc-af99-77bce92c6e20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:54.845799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:54.846769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:54.847563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 198/383] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:54.858660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:55.127378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89333b8-d142-48e8-894a-bcf5632e3cbe None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:55.134270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:55.207582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:55.208804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.209318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.209824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.211912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:55.212476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:55.215339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:55.216351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eae03-d61f-4c0c-8aa6-62a12b9954da None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.216914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eae03-d61f-4c0c-8aa6-62a12b9954da None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.217402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4eae03-d61f-4c0c-8aa6-62a12b9954da None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.219318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 199/384] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:28:55.225168 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.225770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.227394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.234423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 186/385] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:55.243617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:55.248090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:55.249780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:55.345560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 200/386] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:28:55.403671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ab1c-4786-4c05-ae98-61f774a3850f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:55.408313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:55.468447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:55.476407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:55.478924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.480130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.485009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.485009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:55.486785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:55.489718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 187/387] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:55 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:55.498891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:55.575473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:55.634304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ba315-3ed3-448c-86d5-05432daaebc7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:55.713277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677da0-326d-4d65-8c6f-ec95ae0f13f7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:55.726944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:55.727516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.727856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.728205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.761593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:55.763998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:55.765561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.766078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.766557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.768059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:55.769865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:55.772288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 188/388] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:55 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:55.774694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:55.775574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:28:55.776403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:55.779429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 201/389] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:28:55.781307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:55.793536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:55.915490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:55.936800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7087d5-3b2d-47ab-b4f0-18766d6cc038 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:55.993402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:55.994022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:55.994310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:55.994587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:55.995366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:55.995678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:55.997151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:56.006641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 202/390] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:28:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:56.013701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:56.048350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef65f-bf17-479a-ae2b-d9e3dbe7a915 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:56.052837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:56.153354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:56.154647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.155143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.155641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081/groups/c2c5110e97d844ce8670f4be765f561d/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.158079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=059d0856d2f94b658054f1a53c494081, group_id=c2c5110e97d844ce8670f4be765f561d, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:56.159847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:56.229800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a73304-44fb-4037-8ceb-19b25e131365 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:56.259242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 189/391] 10.222.0.58 () {58 vars in 1442 bytes} [Fri May 27 03:28:55 2022] PUT /identity/v3/projects/059d0856d2f94b658054f1a53c494081/groups/c2c5110e97d844ce8670f4be765f561d/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:56.295744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293bb7a-72a9-4b36-874a-ac9fec91514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:56.297404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bb7a-72a9-4b36-874a-ac9fec91514b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.297920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bb7a-72a9-4b36-874a-ac9fec91514b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.298522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293bb7a-72a9-4b36-874a-ac9fec91514b None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.301125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293bb7a-72a9-4b36-874a-ac9fec91514b None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:56.304092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293bb7a-72a9-4b36-874a-ac9fec91514b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:56.307939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 203/392] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:56 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:56.314299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:56.392889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7d7a0-5efa-4bc4-b4e8-663700d6c2db None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.393403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7d7a0-5efa-4bc4-b4e8-663700d6c2db None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.393930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7d7a0-5efa-4bc4-b4e8-663700d6c2db None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.396730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 204/393] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:56.403451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.403764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.404024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.446222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:56.450516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:56.473371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:56.575789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4b295-374b-4da6-8a75-bfa98e97cbb0 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:56.626684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 205/394] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:56.632077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:56.633732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.634225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.634669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.636896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:56.639591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:56.642869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 190/395] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:56 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:56.650000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:56.652934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:56.721671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:56.833498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a0455-5200-47e1-9399-cd93dfddc194 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:56.894723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a7e99-c322-4029-9d42-468e55d9f6c6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:56.941613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:56.942901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.943784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.944810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/groups/c2c5110e97d844ce8670f4be765f561d/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.948418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d, group_id=c2c5110e97d844ce8670f4be765f561d, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:56.950756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:56.951649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:56.952760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:56.953315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:56.953823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:56.974861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:56.987668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:56.988982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:56.990034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 191/396] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:57.002624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:57.034794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 206/397] 10.222.0.58 () {58 vars in 1442 bytes} [Fri May 27 03:28:56 2022] PUT /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/groups/c2c5110e97d844ce8670f4be765f561d/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:57.161148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b41d6-0e98-4a0d-b11e-4a5fc2e36b42 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:57.165116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:57.201664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:57.202710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:57.203171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:57.203624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:57.204726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:57.205243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:57.206896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:57.221700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 192/398] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:57.224280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:57.235829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:57.371215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f670480-de7e-4595-a411-29279a74fa84 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:57.446481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b78324-5a3b-4998-9c58-1ca97eab9b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:57.448315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b78324-5a3b-4998-9c58-1ca97eab9b1f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:57.448930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b78324-5a3b-4998-9c58-1ca97eab9b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:57.449449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b78324-5a3b-4998-9c58-1ca97eab9b1f None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:57.457941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b78324-5a3b-4998-9c58-1ca97eab9b1f None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:57.460966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b78324-5a3b-4998-9c58-1ca97eab9b1f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:57.464579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 207/399] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:57 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:57.470332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:57.732034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbc5a3-ae29-4ece-88ba-8945c869907f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:57.742945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61276584-9436-4670-897b-c4804001284e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:57.743727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61276584-9436-4670-897b-c4804001284e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:57.744214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61276584-9436-4670-897b-c4804001284e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:57.747026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 208/400] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:57.756444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:57.757003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:57.758133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:57.792610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:57.793496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:57.793788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:57.794068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:57.801189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:57.802662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:57.805288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 193/401] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:57 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:57.811213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:57.812560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:57.813796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:57.840096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:58.020541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 209/402] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:58.100989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f6ba1-683c-40c2-8a24-2f5eb1eafd8d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:58.109569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:58.115364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:58.168118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561fd74-230e-48cb-b8e2-dd90751181dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:58.169086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561fd74-230e-48cb-b8e2-dd90751181dd None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:58.169377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561fd74-230e-48cb-b8e2-dd90751181dd None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:58.169661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561fd74-230e-48cb-b8e2-dd90751181dd None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081/groups/c2c5110e97d844ce8670f4be765f561d/roles/c636e98b844543518e09f390c4850083` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:58.172133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561fd74-230e-48cb-b8e2-dd90751181dd None admin] RBAC: Authorizing `identity:create_grant(project_id=059d0856d2f94b658054f1a53c494081, group_id=c2c5110e97d844ce8670f4be765f561d, role_id=c636e98b844543518e09f390c4850083)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:58.173705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561fd74-230e-48cb-b8e2-dd90751181dd None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:58.276819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 194/403] 10.222.0.58 () {58 vars in 1442 bytes} [Fri May 27 03:28:57 2022] PUT /identity/v3/projects/059d0856d2f94b658054f1a53c494081/groups/c2c5110e97d844ce8670f4be765f561d/roles/c636e98b844543518e09f390c4850083 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:28:58.339501 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c5c39-2d3d-4706-8e7f-ea13b5044e8f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:58.419150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:58.420356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c5643-eaa0-4738-bdad-8de84e80c63c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:58.420672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:58.420842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c5643-eaa0-4738-bdad-8de84e80c63c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:58.420960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c5643-eaa0-4738-bdad-8de84e80c63c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:58.421159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:58.421611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:58.423410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 195/404] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:58.428734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:58.429875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:58.430532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:58.452195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:58.463743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:58.464761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:28:58.465569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:28:58.471686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 210/405] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:58.473167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:58.474906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:58.484044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:58.620408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 196/406] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:28:58.623144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:58.654880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:58.672396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:58.712150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42c0ad-87e1-4ca1-9d47-3b18babd67f6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:58.802686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:58.803895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:58.804467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:58.804981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:58.806487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:58.806993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:58.809717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:58.841008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 211/407] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:28:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:28:58.844559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:58.926187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfce78-a96f-4d9a-8c78-28476305dcb4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.014246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.015066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.015354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.015634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.038267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:28:59.059021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:28:59.059977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:28:59.061281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 197/408] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:28:59.078775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:59.161349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:59.174626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd92a8-ee91-4de0-a4d5-9887c0a1c7ac None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.201343 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2fe1c-d56b-4655-b776-d0b61d57d487 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.254765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.256245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.257003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.258608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:59.259212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.262327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:59.263095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:59.266553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:59.285852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 198/409] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:28:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:59.291564 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.293432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.295446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:59.300297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.301257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.308837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:59.315211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:59.327041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 212/410] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:58 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:59.331198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:59.613143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa0e5f-f282-41e6-ac50-9528c31600ec None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.654746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74943-58f7-4458-8548-46b1cbcad537 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.691105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.693394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.694266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.695567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.698545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:59.702085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:59.706785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 213/411] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:59 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:59.714319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:59.751229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:59.763876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0438373-0edb-4c7c-88d4-f61577bb146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.765366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438373-0edb-4c7c-88d4-f61577bb146e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.765882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438373-0edb-4c7c-88d4-f61577bb146e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.766383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438373-0edb-4c7c-88d4-f61577bb146e None admin] PATH_INFO: `/v3/groups/edda4c9ce90a432e92a2b2b21a0965a7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.768960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0438373-0edb-4c7c-88d4-f61577bb146e None admin] RBAC: Authorizing `identity:get_group(group_id=edda4c9ce90a432e92a2b2b21a0965a7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:59.771898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0438373-0edb-4c7c-88d4-f61577bb146e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:59.776154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 199/412] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:28:59 2022] GET /identity/v3/groups/edda4c9ce90a432e92a2b2b21a0965a7 => generated 229 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:59.778630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40cc7-2c3a-4d28-9bf4-a392fc2a60ad None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.779120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40cc7-2c3a-4d28-9bf4-a392fc2a60ad None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.786638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d40cc7-2c3a-4d28-9bf4-a392fc2a60ad None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:59.790451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 200/413] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:28:59 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:59.795827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40cc7-2c3a-4d28-9bf4-a392fc2a60ad None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:59.862669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:59.938005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f21fe1-d33a-4029-b402-9e98c62a14bd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.970568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d40cc7-2c3a-4d28-9bf4-a392fc2a60ad None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:59.996934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:59.998404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:59.998900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:59.999918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/groups/c2c5110e97d844ce8670f4be765f561d/roles/c636e98b844543518e09f390c4850083` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.003435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d, group_id=c2c5110e97d844ce8670f4be765f561d, role_id=c636e98b844543518e09f390c4850083)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:00.006045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:00.029216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db4b14-d3f7-4c72-83a5-5d9cdc80f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:00.030326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db4b14-d3f7-4c72-83a5-5d9cdc80f693 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.030712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db4b14-d3f7-4c72-83a5-5d9cdc80f693 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.032368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db4b14-d3f7-4c72-83a5-5d9cdc80f693 None admin] PATH_INFO: `/v3/projects/1781b8350b784207b421bac2bc80c855` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.032570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db4b14-d3f7-4c72-83a5-5d9cdc80f693 None admin] RBAC: Authorizing `identity:get_project(project_id=1781b8350b784207b421bac2bc80c855)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:00.034194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db4b14-d3f7-4c72-83a5-5d9cdc80f693 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:00.036131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 201/414] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:28:59 2022] GET /identity/v3/projects/1781b8350b784207b421bac2bc80c855 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:00.040764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.040764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.040764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.071560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:00.073602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:00.121438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 214/415] 10.222.0.58 () {58 vars in 1442 bytes} [Fri May 27 03:28:59 2022] PUT /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d/groups/c2c5110e97d844ce8670f4be765f561d/roles/c636e98b844543518e09f390c4850083 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:00.123994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:00.190089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:00.200244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 202/416] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:00.236450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:00.274883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7371d-88ed-4f4b-8958-0d03bca9feb9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:00.308979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:00.318301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62310f1e-0b01-4aa2-8044-ff66728ff7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:00.319217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62310f1e-0b01-4aa2-8044-ff66728ff7c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.319534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62310f1e-0b01-4aa2-8044-ff66728ff7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.320625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62310f1e-0b01-4aa2-8044-ff66728ff7c6 None admin] PATH_INFO: `/v3/projects/1781b8350b784207b421bac2bc80c855/groups/edda4c9ce90a432e92a2b2b21a0965a7/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.322857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62310f1e-0b01-4aa2-8044-ff66728ff7c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1781b8350b784207b421bac2bc80c855, group_id=edda4c9ce90a432e92a2b2b21a0965a7, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:00.324359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62310f1e-0b01-4aa2-8044-ff66728ff7c6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:00.425157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0bc08f-d83c-4c84-be0b-44c8979551b1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:00.435554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 215/417] 10.222.0.58 () {58 vars in 1442 bytes} [Fri May 27 03:29:00 2022] PUT /identity/v3/projects/1781b8350b784207b421bac2bc80c855/groups/edda4c9ce90a432e92a2b2b21a0965a7/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:00.437084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7b6f1-c1a6-408d-ba9c-2715a0fd942c None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.438374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7b6f1-c1a6-408d-ba9c-2715a0fd942c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.438374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7b6f1-c1a6-408d-ba9c-2715a0fd942c None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.439993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 216/418] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:00.447024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.447024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.447024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.467567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:00.468944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:00.495118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:00.497337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.497635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.497917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.520880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:00.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:00.540628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:29:00.541417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:00.547036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 203/419] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:00.557263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:00.614307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 217/420] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:00.675275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:00.686079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:00.732730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7194c-8965-41df-b3e6-c9290b2514f3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:00.801829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:00.806071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.806865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.807423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.809066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] RBAC: Adding query filter params (name=member) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:00.809725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:00.812766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:00.828631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 204/421] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:29:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:00.830554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:00.860853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc7e86-9e87-43f2-b5dd-ffa78f8a3e2c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:00.882215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:00.911574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:00.914427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.918624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.919956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.942983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:00.964600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:00.965738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:29:00.966718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:00.970074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 218/422] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:00.978703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.122438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561741ad-bfa9-4875-9dc1-debc8b30c516 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.147356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:01.149627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a970a46-0e42-46ed-a456-b51751a553ca None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.166103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b435f-610b-4cf3-94b0-5bb6b0db8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.166879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b435f-610b-4cf3-94b0-5bb6b0db8251 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.167237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b435f-610b-4cf3-94b0-5bb6b0db8251 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.167586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b435f-610b-4cf3-94b0-5bb6b0db8251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.198049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.199138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.199914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.200644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.201662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:01.202444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:01.204842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da501b-6eec-4177-9b9d-451db9f6cbd0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:01.223239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 219/423] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:29:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:01.223239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243fff-24d9-4646-aaf3-1c9176143f60 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.223239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243fff-24d9-4646-aaf3-1c9176143f60 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.223239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b243fff-24d9-4646-aaf3-1c9176143f60 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.224364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 220/424] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:01.227731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b243fff-24d9-4646-aaf3-1c9176143f60 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.261477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b435f-610b-4cf3-94b0-5bb6b0db8251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:01.264330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b435f-610b-4cf3-94b0-5bb6b0db8251 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:01.280623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:01.362471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b243fff-24d9-4646-aaf3-1c9176143f60 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.405876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.406581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.407175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.407558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.414794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 205/425] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:29:01.414794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.415582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.415582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.434375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:01.435734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:01.453778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:01.505217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:01.508069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:01.519482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 206/426] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:01.520824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.560684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:01.648433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 221/427] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:01.650412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.665366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bede2-b91e-45cf-ba40-34d2e8f3e572 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.708563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.709681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.710851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.711968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.713641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:01.715479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:01.722636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 207/428] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:01.722636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.750102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:01.795141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d8bbf-1a6c-445f-96a8-e86edf8cf546 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.844367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.845677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.846134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.846910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.849688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:01.859237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc632f8-1434-471b-8a58-83fecf607465 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:01.863535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:01.878391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:01.879441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:29:01.880716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:01.883357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 222/429] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:01.884130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:01.916152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:01.917074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:01.917379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:01.917716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:01.919554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:01.921092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:01.923531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 208/430] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:01.924795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:02.035328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f11b8-2360-40df-8b13-4d8f45d15643 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:02.043144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:02.063734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b13dc6-a94f-4dc3-8334-8ede61b988b3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:02.084065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:02.085557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:02.086549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:02.086835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:02.088087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:02.089737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:02.091923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 223/431] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:02.092858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:02.105490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:02.106173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:02.106462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:02.106741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:02.108059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:02.108377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:02.109802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:02.122032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 209/432] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:02.122718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:02.181869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:02.232156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df0f75-8f9d-42c2-b012-d2e09f129360 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:02.261539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9a33e-723b-4a93-a524-555431057153 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:02.284180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:02.285766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:02.286759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:02.287268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:02.289864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:02.292155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:02.299975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 224/433] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:29:02 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:02.299975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:02.310436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:02.311108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:02.311398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:02.311789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:02.313015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=059d0856d2f94b658054f1a53c494081) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:02.313606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:02.315553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:02.324848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 210/434] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:29:02 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:02.326399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:02.349325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:02.493945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdad9d3-395a-4140-8478-96f5572d9d1f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:02.509258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0825644-d5ae-47af-aac1-e2510e3301d9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:02.552285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bc352-a071-4ab4-944d-d28f855b73f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:02.553022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bc352-a071-4ab4-944d-d28f855b73f5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:02.553312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bc352-a071-4ab4-944d-d28f855b73f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:02.553593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bc352-a071-4ab4-944d-d28f855b73f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:02.558490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:02.582995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:02.584053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:02.584935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:02.586816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:02.588005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=93828c8fe28c41ca831b85d55b1a50ea, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:02.588763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:02.590717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:02.599980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 211/435] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:29:02 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=93828c8fe28c41ca831b85d55b1a50ea => generated 611 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:02.706465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bc352-a071-4ab4-944d-d28f855b73f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:02.708589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bc352-a071-4ab4-944d-d28f855b73f5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:02.844316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 225/436] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:02.850769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:02.964240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:02.994884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda440d-bc20-4024-86c1-744a5aaa71bc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.055821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408908-d452-48f4-9933-791918c4e289 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.056169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408908-d452-48f4-9933-791918c4e289 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.056462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408908-d452-48f4-9933-791918c4e289 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.058175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 226/437] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:03.063436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.065048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.065512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.065638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.066079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] PATH_INFO: `/v3/projects/059d0856d2f94b658054f1a53c494081` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.066538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.067201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.068539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] RBAC: Authorizing `identity:get_project(project_id=059d0856d2f94b658054f1a53c494081)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:03.071636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:03.076491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 212/438] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:29:02 2022] GET /identity/v3/projects/059d0856d2f94b658054f1a53c494081 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:03.083794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.094919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:03.096631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:03.149318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:03.227309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e5c5b8-d27f-4e6b-8a7d-8559c51e04cc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 227/439] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:03.258956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.279400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.280364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.280688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.281010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.282366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:03.284216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:03.286054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 213/440] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:03.292790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.312003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:03.392623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e4a75-310e-41e1-9417-c6b3de21a068 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.429756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da846e98-0b84-4a55-a256-e3c2a2ed374a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.452789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.454095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.454554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.455008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.473544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:03.487629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:03.489463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:03.496329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 228/441] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:03.497185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.497185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.497185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.497185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.497481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=c636e98b844543518e09f390c4850083, scope.project.id=059d0856d2f94b658054f1a53c494081) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:03.497481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:03.497481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301efe34-6f03-45b6-a039-240336d35ad0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:03.499254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.505740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 214/442] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=059d0856d2f94b658054f1a53c494081&role.id=c636e98b844543518e09f390c4850083 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:03.530486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:03.623284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5a17-e56c-4cb8-8d61-b5f0fbfade02 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.623694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5a17-e56c-4cb8-8d61-b5f0fbfade02 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.623891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5a17-e56c-4cb8-8d61-b5f0fbfade02 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.625655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 215/443] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:29:03.631306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.631616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.631712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72ef82-4083-45c0-b3f8-704e72c90ecc None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.632523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.651673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:03.652889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:03.678535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.680104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.680585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.680869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.682258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:03.682586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:03.684525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:03.701659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 229/444] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:03.703154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.734963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 216/445] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:03.759996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.768897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:03.804456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:03.832011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daac2a3-51bd-4bc4-9eaf-307bef77c3b8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.881371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.883403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.883683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723a7fc-76de-4bfd-8f70-0d1b76074583 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:03.884787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.885210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.930163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:03.930975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:03.931259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:03.931531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:03.942535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:03.950559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:03.951520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:03.952421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 217/446] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:03.957919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:03.985626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:03.987509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.050061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.098332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842de57-81f3-4158-8401-5b1ff47a0935 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.103035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.127171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 230/447] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:29:04.134388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.138315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.138998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.139281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.139558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.140319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:04.140659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.142106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.151336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 218/448] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:04.157093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.245924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b1644-b453-49ea-9c5e-54f27864d173 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.270462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b501e0-fad3-4e27-b00d-46795bff15b6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.283873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.284734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.285011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.285279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] PATH_INFO: `/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.287465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] RBAC: Authorizing `identity:get_project(project_id=82a6bbe486fe4eac8bf7bfd0072b8d6d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.289347 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.291129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 231/449] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/projects/82a6bbe486fe4eac8bf7bfd0072b8d6d => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:04.296868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.299655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.310022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.312665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.312665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.312665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.351518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.388670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.390154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.405716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e96ac8-87cf-4d24-a036-fb9c58ae1fce None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.442575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.443433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.443728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.443992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] PATH_INFO: `/v3/groups/c2c5110e97d844ce8670f4be765f561d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.445415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] RBAC: Authorizing `identity:get_group(group_id=c2c5110e97d844ce8670f4be765f561d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.446885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.448572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 232/450] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/groups/c2c5110e97d844ce8670f4be765f561d => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:04.454406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.500703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 219/451] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:29:04.506644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.535143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.571380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a8db6-246c-416e-acbf-30750f7b567a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.600445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.620886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.621798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.622350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.622810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.623984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] RBAC: Adding query filter params (group.id=c2c5110e97d844ce8670f4be765f561d, role.id=c636e98b844543518e09f390c4850083, scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:04.624296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.624872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b59245-5983-498e-b151-fc3b7861b1be None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.625818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.632403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 233/452] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/role_assignments?group.id=c2c5110e97d844ce8670f4be765f561d&scope.project.id=82a6bbe486fe4eac8bf7bfd0072b8d6d&role.id=c636e98b844543518e09f390c4850083 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:04.664707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.665608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.665887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.666162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] PATH_INFO: `/v3/projects/1781b8350b784207b421bac2bc80c855` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.667503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] RBAC: Authorizing `identity:get_project(project_id=1781b8350b784207b421bac2bc80c855)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.669048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.670861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 220/453] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/projects/1781b8350b784207b421bac2bc80c855 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:04.679392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.815703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120463e-4db3-45bb-bcd9-fc014a314d19 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:04.853869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cb246-e85b-43c2-a270-1e2ab61f5d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:04.854740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cb246-e85b-43c2-a270-1e2ab61f5d32 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:04.855029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cb246-e85b-43c2-a270-1e2ab61f5d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:04.855333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cb246-e85b-43c2-a270-1e2ab61f5d32 None admin] PATH_INFO: `/v3/groups/edda4c9ce90a432e92a2b2b21a0965a7` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:04.856603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044cb246-e85b-43c2-a270-1e2ab61f5d32 None admin] RBAC: Authorizing `identity:get_group(group_id=edda4c9ce90a432e92a2b2b21a0965a7)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:04.858080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044cb246-e85b-43c2-a270-1e2ab61f5d32 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:04.859757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 234/454] 10.222.0.58 () {56 vars in 1180 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/groups/edda4c9ce90a432e92a2b2b21a0965a7 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:04.865659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:04.990442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:04.997688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ddca0-558e-4ac7-b461-6b47afe781b9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:05.039652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:05.040323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:05.040652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:05.040936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:05.041778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] RBAC: Adding query filter params (group.id=edda4c9ce90a432e92a2b2b21a0965a7, role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=1781b8350b784207b421bac2bc80c855) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:05.042106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:05.043579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7139-03d6-4afb-98c3-00e76ceb9c6a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:05.050275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 221/455] 10.222.0.58 () {56 vars in 1376 bytes} [Fri May 27 03:29:04 2022] GET /identity/v3/role_assignments?group.id=edda4c9ce90a432e92a2b2b21a0965a7&scope.project.id=1781b8350b784207b421bac2bc80c855&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:08.212104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28034ba-cf64-49ca-b897-a88394d9938b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.213049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28034ba-cf64-49ca-b897-a88394d9938b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.213488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28034ba-cf64-49ca-b897-a88394d9938b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.215050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 235/456] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:29:08.223196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590853e-264f-42c2-bcb4-5153ecf37619 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.224063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590853e-264f-42c2-bcb4-5153ecf37619 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.224493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590853e-264f-42c2-bcb4-5153ecf37619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.258416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b590853e-264f-42c2-bcb4-5153ecf37619 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:08.259884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b590853e-264f-42c2-bcb4-5153ecf37619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:08.355505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 222/457] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:08.433826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28034ba-cf64-49ca-b897-a88394d9938b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:08.500920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:08.516561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179c4f8-1389-460a-b028-486016a6e1c8 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.516561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179c4f8-1389-460a-b028-486016a6e1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.516561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179c4f8-1389-460a-b028-486016a6e1c8 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.517836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 223/458] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:08.527392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6720ae1-a85d-4df1-98dc-f5bb7a477885 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.527764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6720ae1-a85d-4df1-98dc-f5bb7a477885 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.528061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6720ae1-a85d-4df1-98dc-f5bb7a477885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.565475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c6720ae1-a85d-4df1-98dc-f5bb7a477885 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:08.566366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28034ba-cf64-49ca-b897-a88394d9938b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:08.566693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c6720ae1-a85d-4df1-98dc-f5bb7a477885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:08.619033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:08.620114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.620595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.621037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.636329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:08.639484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:08.665207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 224/459] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:08.666558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceaa65-adeb-43a0-8a0a-e8d1021c45a6 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.666844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceaa65-adeb-43a0-8a0a-e8d1021c45a6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.667120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceaa65-adeb-43a0-8a0a-e8d1021c45a6 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.668398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 225/460] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:08.681618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:08.683329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:08.684969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:08.684969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:08.684969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:08.684969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:08.687140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.687433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.687715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.688443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 236/461] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:08.697492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:08.716850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:08.719133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:08.740540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:08.821843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 226/462] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:08.823277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:08.833276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140196e-5fb5-4024-9f94-a4a4530f3a81 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:08.835931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:08.885853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:08.886767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.887210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.887639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:08.889143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:08.889657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:08.891239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:08.910407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 237/463] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:08.911644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:08.956156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaa533-a45d-4c3a-825a-e12148803348 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:08.995714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:08.997082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:08.997217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:08.997617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.010292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.012262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.035258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0e9b4-0748-4b13-a123-d652ed96f8e1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:09.035732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:09.036763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:09.037854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:09.037854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:09.037854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:09.037854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:09.037854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 227/464] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:09.038586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.048196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:09.091403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.093562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.094012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.094960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.114325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.117988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:09.149880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:09.151196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:09.152603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:09.152603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:09.152603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:09.152603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:09.152603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 238/465] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:09.152603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.171756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecab06b-aba7-4921-a18a-c4a069ee07b0 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.181286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:09.223328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.225921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.226643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.228229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.229729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.232406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.300242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89338b9e-ebc6-49ef-a612-2e6d1457c6b9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.336540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 228/466] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:29:09 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:09.336540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.370810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.372790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.373075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.373352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.375001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:09.375376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.376840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d14df-f3e0-44c4-85e5-70d3563de17c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.390802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 239/467] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:09.392183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8faa37-01d0-41a6-aaf5-08ab65bff642 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.392470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8faa37-01d0-41a6-aaf5-08ab65bff642 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.398187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8faa37-01d0-41a6-aaf5-08ab65bff642 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.405033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 240/468] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:09.405033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8faa37-01d0-41a6-aaf5-08ab65bff642 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.456937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:09.497080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32857ba-4966-4191-be56-a3c6ec5bd97a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.536922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.537615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.538491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.538921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.539680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:09.540021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.541821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaac0ea-0995-4244-af05-295fe88b8987 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.554419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:09.561206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 229/469] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:09.561206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de100e1-14d6-4f96-934c-639b92265c00 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.561206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de100e1-14d6-4f96-934c-639b92265c00 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.561206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de100e1-14d6-4f96-934c-639b92265c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.577437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7de100e1-14d6-4f96-934c-639b92265c00 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:09.578487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7de100e1-14d6-4f96-934c-639b92265c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:09.593530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8faa37-01d0-41a6-aaf5-08ab65bff642 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.644938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb013d4d-fa00-466e-a418-193e93bfeae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.645538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb013d4d-fa00-466e-a418-193e93bfeae9 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.645819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb013d4d-fa00-466e-a418-193e93bfeae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.646096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb013d4d-fa00-466e-a418-193e93bfeae9 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.646866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb013d4d-fa00-466e-a418-193e93bfeae9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.649191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb013d4d-fa00-466e-a418-193e93bfeae9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.673531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 230/470] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:09.674876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5c0aa-7d58-4b2c-b7b2-4096cc99dc62 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.675171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5c0aa-7d58-4b2c-b7b2-4096cc99dc62 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.675454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5c0aa-7d58-4b2c-b7b2-4096cc99dc62 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.680574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 231/471] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:09.680574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5c0aa-7d58-4b2c-b7b2-4096cc99dc62 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.740342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:09.743871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 241/472] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:29:09 2022] POST /identity/v3/users => generated 246 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:09.745367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.745683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.745986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.765541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:09.769021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:09.825061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5c0aa-7d58-4b2c-b7b2-4096cc99dc62 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:09.869068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 242/473] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:09.869966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:09.915427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:09.916673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:09.917155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:09.917766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:09.919186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:09.929326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:09.953260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:10.020472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74011-8a78-43a4-8fa3-8e1690ef7ae6 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:10.068598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 232/474] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:29:09 2022] POST /identity/v3/users => generated 248 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:10.071169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:10.081988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd81489-896b-429c-a671-2069cd291add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:10.083452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd81489-896b-429c-a671-2069cd291add None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:10.083899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd81489-896b-429c-a671-2069cd291add None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:10.084997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd81489-896b-429c-a671-2069cd291add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:10.111390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd81489-896b-429c-a671-2069cd291add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:10.115391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd81489-896b-429c-a671-2069cd291add None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ccd81489-896b-429c-a671-2069cd291add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:10.160821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:10.162359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:10.163977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:10.163977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:10.163977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:10.163977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:10.163977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 243/475] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:10.181538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd81489-896b-429c-a671-2069cd291add None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:10.259290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dcfcb-b8cf-414c-aaeb-4deb9cec8334 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:10.262453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:10.315872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:10.316704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:10.316993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:10.317274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:10.318059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:10.319751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:10.364056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.sql.core [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:29:10.364056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:29:10.364056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [parameters: {'id': '23f60bd2115a4643a202971f7ea159c9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:29:10.364056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80700) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application conn.query(q) May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result.read() May 27 03:29:10.395295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return ref.to_dict() May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:29:10.396845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application next(self.gen) May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application yield resource May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application next(self.gen) May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise self.value May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application session.commit() May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:29:10.398653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.session.flush() May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application compat.raise_( May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:29:10.400235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:29:10.409669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application conn.query(q) May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result.read() May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:29:10.413687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [parameters: {'id': '23f60bd2115a4643a202971f7ea159c9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:10.416919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:29:10.420187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:10.428038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 233/476] 10.222.0.58 () {60 vars in 1131 bytes} [Fri May 27 03:29:10 2022] POST /identity/v3/roles => generated 144 bytes in 335 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:10.428038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:10.445939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd81489-896b-429c-a671-2069cd291add None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:10.478411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:10.521485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:10.523182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:10.523949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:10.524771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:10.526321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:10.528774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:10.531708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:10.557251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 244/477] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:10.566068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:10.646101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741d60e-a319-4550-a90b-14c652fad57b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:10.729325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:10.730545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:10.731039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:10.731615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:10.754786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:10.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:10.769499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:29:10.770295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:10.776035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 234/478] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:29:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:10.782827 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af83338-d96c-4a5c-8ddc-8a9bb5613d8a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:10.786763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:10.847736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:10.869644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f941-5e53-4a7d-8f1f-057a73eb0d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:10.870689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f941-5e53-4a7d-8f1f-057a73eb0d1c None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:10.871204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f941-5e53-4a7d-8f1f-057a73eb0d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:10.871708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f941-5e53-4a7d-8f1f-057a73eb0d1c None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:10.877715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f941-5e53-4a7d-8f1f-057a73eb0d1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:10.877715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f941-5e53-4a7d-8f1f-057a73eb0d1c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:10.964755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a86f3-7342-4ed1-ac5a-490197ad3eff None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:10.981931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 245/479] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:29:10 2022] POST /identity/v3/users => generated 249 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:11.006911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:11.007664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:11.008394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:11.008717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:11.009527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:11.010242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:11.012298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b73ee-54f2-459b-886c-6deeb1421c9d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:11.022559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 235/480] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:12.454637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dcf38-3f8c-4ae2-8d9d-c63ddf1ab260 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:12.455193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dcf38-3f8c-4ae2-8d9d-c63ddf1ab260 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:12.455493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dcf38-3f8c-4ae2-8d9d-c63ddf1ab260 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:12.457828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 246/481] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:12.465117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7589e-46db-44e3-8827-f7580eb2d1a2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:12.465614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7589e-46db-44e3-8827-f7580eb2d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:12.465988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7589e-46db-44e3-8827-f7580eb2d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:12.496549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4df7589e-46db-44e3-8827-f7580eb2d1a2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:12.498231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4df7589e-46db-44e3-8827-f7580eb2d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:12.667226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 236/482] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:12.706859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dcf38-3f8c-4ae2-8d9d-c63ddf1ab260 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:12.742641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee536-3e11-4382-ab45-797e28e77de7 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:12.743296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee536-3e11-4382-ab45-797e28e77de7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:12.743780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee536-3e11-4382-ab45-797e28e77de7 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:12.747728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 237/483] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:29:12.753000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:12.753586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:12.754092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:12.774650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:12.776226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:12.801576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:12.824881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:12.973355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 238/484] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:13.029250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.044792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dcf38-3f8c-4ae2-8d9d-c63ddf1ab260 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.155942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.156792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.157072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.157341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.181771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:13.190170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:13.191124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:13.191914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:13.199870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 247/485] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 490 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:13.201528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.255338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1ee91-42f7-48d3-8eee-c77a9a953772 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.306825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8da99-228e-49c8-becf-4c7af148ac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.307647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8da99-228e-49c8-becf-4c7af148ac2e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.307935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8da99-228e-49c8-becf-4c7af148ac2e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.308213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8da99-228e-49c8-becf-4c7af148ac2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.320298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8da99-228e-49c8-becf-4c7af148ac2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-4ad8da99-228e-49c8-becf-4c7af148ac2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:13.328339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:13.329371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:13.330797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 239/486] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:13.331478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6e903-47c3-414a-ba6e-161c15f4697b None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.331543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6e903-47c3-414a-ba6e-161c15f4697b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.331794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6e903-47c3-414a-ba6e-161c15f4697b None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.333257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 240/487] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:13.337415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6e903-47c3-414a-ba6e-161c15f4697b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.353122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:13.355305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:13.370051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dee3a-1c40-4f0c-b1f9-e1cea9efe8d7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.421739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.422461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.422749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.423079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.423836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:13.424144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:13.425870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eba574-aea0-405a-bab5-cbb9bd11e9fc None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:13.441535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 248/488] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:13.443148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.443447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.443766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.464382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:13.465848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:13.490367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6e903-47c3-414a-ba6e-161c15f4697b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.533063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.534031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.534512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.534986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.535934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:13.536479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:13.539107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:13.549046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 249/489] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:13.549974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 241/490] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:13.550225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.560645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.643621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:13.667372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e4d32-a31c-4a4e-bdd7-f84a7d38cb73 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.688005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:13.709536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.710741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.710850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.711088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.724069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:13.726686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:13.739022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01865fc5-fa31-4ffd-beae-b54f41ab6f74 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:13.747737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:13.749271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:13.750836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:13.750836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:13.750836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:13.750836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:13.753837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 250/491] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:13.754792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.799746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.800527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.800816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.801098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:13.814471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:13.815887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:13.846975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:13.847902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:13.848756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:13.848756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:13.848756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:13.848756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:13.856654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 242/492] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:13.858218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:13.921502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad897-2a82-4570-8cb3-7c2337afab41 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:13.982250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:13.988613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:13.989606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:13.989890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:13.990165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.003864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:14.012517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:14.013512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:14.014320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:14.019643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 251/493] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:14.020624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.053824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a32bbf-31c4-4146-a603-4d7669f8a0ce None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.086740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:14.100941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.101964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.102468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.102949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.103873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:14.104392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.106015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.120362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 243/494] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:14.121586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.142984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af108336-2157-43bd-ac47-c9b1f3507da6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.179997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.180713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.180998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.181273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.182006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:14.182308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.183840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.200747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 252/495] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:14.201767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.245196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15e07e-9a07-4c34-b365-87753c178ac9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.270104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:14.284117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672149-fde0-46c5-b303-077003677440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.284825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672149-fde0-46c5-b303-077003677440 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.285114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672149-fde0-46c5-b303-077003677440 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.285401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672149-fde0-46c5-b303-077003677440 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.286169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672149-fde0-46c5-b303-077003677440 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:14.286480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672149-fde0-46c5-b303-077003677440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.288007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672149-fde0-46c5-b303-077003677440 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.299684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 244/496] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:14.302162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa6c1-32a4-42d4-86f8-463dff969b0a None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.302672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa6c1-32a4-42d4-86f8-463dff969b0a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.303796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa6c1-32a4-42d4-86f8-463dff969b0a None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.307451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 245/497] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:14.308901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffa6c1-32a4-42d4-86f8-463dff969b0a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.345775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48fe0-e7e8-4b83-be00-c4bc75509100 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.383554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:14.404279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.407549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.407549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.407549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.420375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.422392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:14.441096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:14.442154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:14.443060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 253/498] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:14.443671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.452144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffa6c1-32a4-42d4-86f8-463dff969b0a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.494511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.495619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.496079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.496563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.509664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.512113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a273c9bc-b9d4-4ebf-adf3-46d61ffc3041 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:14.535272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:14.536231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:14.537264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 246/499] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:14.539380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.539380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.539762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.562342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:14.563470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:14.569892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646727e7-082f-48c4-bc00-2c289442489e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.583521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:14.604928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.605733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.606020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.606298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.618038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.619545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-6679d6e7-e7db-4604-8cf6-e5a0afb40bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:14.638591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:14.639683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:14.641062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:14.641062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:14.641062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:14.641062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:14.641062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 254/500] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:14.641514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b767afc-4d8c-4796-b02c-42e1821df052 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.642117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b767afc-4d8c-4796-b02c-42e1821df052 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.642399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b767afc-4d8c-4796-b02c-42e1821df052 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.643714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 255/501] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:14.644650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b767afc-4d8c-4796-b02c-42e1821df052 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.655804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 247/502] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:14.656703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.712362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:14.783613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b767afc-4d8c-4796-b02c-42e1821df052 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.789707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe634c1-249a-4fc1-87e4-0e8a171b3618 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:14.821089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.821679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.821960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.822234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.823066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:14.823369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.824849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.825445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:14.826364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.826698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.826985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.827835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:14.828144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:14.829963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba94480-e6ce-4fa7-bd0b-8e7454771bb0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:14.874675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 256/503] 10.222.0.58 () {56 vars in 1137 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:14.874675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:14.884132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 248/504] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:14.885624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:14.885969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:14.886253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:14.906185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:14.906399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:14.907545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:14.998835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a8fd3-a051-42d4-98a1-4d0dce01ccad None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.001542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 249/505] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:15.002711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.034218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.057577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.058261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.058543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.059631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.059631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:15.059870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.061340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.080028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 257/506] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:15.080947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.154829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df6e8fd-6112-4259-a967-a97a199d83f8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.192353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.194045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.194965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.195254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.195528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.206707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.208910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.213714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fad0e-5c0c-4fde-84fc-3d9a954bda7e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:15.229406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:15.230479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:15.231495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:15.231495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:15.231495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:15.231495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:15.231495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 250/507] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:15.231775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.257893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.258555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.258841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.259117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.321836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.341009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.342535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.349415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea4e05-1324-4807-b772-ed88cd44a3d7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.392980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.393918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.394362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.394824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.446719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.466311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 258/508] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:15.467402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.481279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.482760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.579400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558be5b0-f748-4166-8537-e73e65d9c42c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.582333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.613507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 251/509] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:15.613507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.638511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.639633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.640034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.640456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.657234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:15.672922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:15.673903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:15.674716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:15.675475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 259/510] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:15.676547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.722571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57282f90-0af7-42b1-b27c-68505dbf9aa1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.735848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.763827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.764707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.764992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.765269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.773227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.774736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:15.790356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:15.791603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 252/511] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:15.793243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.808719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402f82b-5fca-44b0-9606-f8d8546ae385 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.837469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.849821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.856543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.856543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.856543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.856543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:15.856543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.856543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.868290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 260/512] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:15.869187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.919355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0a7f6-ec6d-4a51-b4e1-0327cf4264fd None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.959060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.960593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.960884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.961161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.972344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.974343 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.983574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ed22-1179-4299-9c87-f8143fadce75 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.985549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:15.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:16.003525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:16.005239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:16.005239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:16.005239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:16.005239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:16.005239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 253/513] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:16.005239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.029478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.030420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.030717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.030995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.043643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.045984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:16.065418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:16.067031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:16.069855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:16.069855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:16.069855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:16.069855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:16.069855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 261/514] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:16.069855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.116341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:16.131115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7a3f9-d62c-4620-b590-2647e21a5cbf None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.171031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.171734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.172017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.172292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.173077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:16.173381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.174787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.184676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 254/515] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:16.185500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.187439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1351f-ddbf-427a-bd56-c3729d6e2baa None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.233586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.234269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.234556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.234836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.235686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:16.236000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.237507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.258429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 262/516] 10.222.0.58 () {56 vars in 1137 bytes} [Fri May 27 03:29:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:16.259288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.276174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:16.319966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ac5dd-f275-4076-b152-cdc334724cf3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.361968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.362852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.363234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.363699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.364756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.366811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.370976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c39cd2-5b49-46f1-a9af-1b2c1b2ff1e8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.409626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.410198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.410480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.410755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.411486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:16.411795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.413429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.425461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 263/517] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:16.426709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.452255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 255/518] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:29:16 2022] POST /identity/v3/users => generated 247 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:16.453716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.470318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:16.523383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:16.561605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c83909-a698-402c-8068-a26f72d0b89e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.617580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.618657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.619190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.619607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.620732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:16.621245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.623006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.632540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb3e71-b922-459d-9fe5-adf0da1dbe40 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.646665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 264/519] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:16.646665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.688885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.690035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.690496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.690952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.710304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.712942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:16.738946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:16.741255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:16.742311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:16.742311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:16.742311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:16.742311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:16.743651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 256/520] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:16.744597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:16.809539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdb857-4660-4e3f-8b53-67c313458144 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.842842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:16.858371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:16.865012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.866541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.866984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.867444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.905117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f2ac5-27c8-4524-aee4-fb2a80d88627 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:16.972272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:16.973046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:16.973414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:16.973705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:16.982899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.983356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:16.984401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:16.985688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:17.001052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:17.005103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 257/521] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:17.005965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.132010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 265/522] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:17.133377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.168666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccb18c-218c-49c1-9114-8b890ffa33a4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.183475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:17.194841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:17.217650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.219321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.219903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.220358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.233364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.235903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:17.261953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:17.262860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:17.263681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 258/523] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:17.264156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.297106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ae532-a010-4709-8c35-f6ef10d0f910 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.359087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.360050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.360314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.360616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.361652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:17.362083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.363795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.384559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 266/524] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:17.384559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.398502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2569a54-ff7d-4bfe-896c-224ac1c59b39 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.447744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.448414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.448754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.449158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.449996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:17.450305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.451746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.476656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 259/525] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:17.476656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.506800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:17.523033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:17.526567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5732e-2201-4f36-bc00-42def39086b3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.570226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.571395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.571918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.572396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.584744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.586487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.597926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af074749-50a4-4e3c-bca4-0338deedb5ef None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:17.605804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:17.607823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:17.609916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:17.609916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:17.609916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:17.609916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:17.609916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 267/526] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:17.609916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.657538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.659611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.660210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.660814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.663100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:17.664131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.666884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.682423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 260/527] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:17.683338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.763659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aa861-3f00-46b4-baa5-ef87efafc521 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.815766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.816814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.817238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.818315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.827217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.828821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.833656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:17.841560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7247b-d72e-426e-865f-aaea89170503 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:17.845463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:17.847411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 268/528] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.871867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:17.896486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.897718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.898002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.898274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.899112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=dc6f327db350476e9e5194719da4cbb5) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:17.899437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.900942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.907275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 261/529] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/role_assignments?user.id=dc6f327db350476e9e5194719da4cbb5&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:17.907981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:18.015920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a1786-f41e-4f33-b3a0-e0194f07f65f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:18.065387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb18896-0dd1-41c5-af90-7a637194ddf4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:18.082441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:18.083755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:18.085482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:18.086231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:18.087749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:18.088461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:18.090927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:18.111228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:18.112041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:18.112391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:18.113003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:18.114123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=0ceb759c7b4b4292bb9763574d79254b) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:18.114540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:18.116817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:18.122254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 269/530] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:18.122254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:18.127661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 262/531] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/role_assignments?user.id=0ceb759c7b4b4292bb9763574d79254b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:18.132561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:18.193758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:18.298943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:18.329141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdd714-3f7b-4649-b7b3-4e82c9e09ff1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:18.402839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800928a7-1187-41b4-9cc3-9473e457353a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:18.431893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:18.434084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:18.434943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:18.435870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:18.446015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:18.446463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:18.446719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:18.446986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:18.447841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:18.448146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:18.449660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:18.463024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:18.466162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:18.492562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 270/532] 10.222.0.58 () {56 vars in 1139 bytes} [Fri May 27 03:29:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:18.492782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:18.494002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:18.495465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 263/533] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:18.500419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:18.503277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:18.779536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:18.801902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258c31-289a-4475-9128-3ff39a999149 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:18.817701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480d076-cf04-4893-944e-d7dbd4147934 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:18.907343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:18.914317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:18.915438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:18.915924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:18.916409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:18.922245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:18.922800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:18.930337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:18.942593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:18.944102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:18.944978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:18.945577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:18.964334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 264/534] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:18.970467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.056158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.057868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.103209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10eaa6-4739-4386-b57b-ec8f00d93ad8 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.142166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.142834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.143120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.143500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.144347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=cd4fb121b8de406899d5698c84d0903b) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:19.144693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.146291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228225e-94b1-497a-9e0a-23d4c174bf49 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.153616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 265/535] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:18 2022] GET /identity/v3/role_assignments?user.id=cd4fb121b8de406899d5698c84d0903b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:19.154924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c400f-af3d-418c-96dd-51d6b73fd4fa None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.155213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c400f-af3d-418c-96dd-51d6b73fd4fa None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.155492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c400f-af3d-418c-96dd-51d6b73fd4fa None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.157032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 266/536] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:19.163550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.164012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.164301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.208800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:19.210923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:19.218092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:19.285062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 271/537] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 781 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:19.295370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.353606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:19.356406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 267/538] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:19.397627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.489802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46cace-ac2a-4b4e-97b0-e033307d8cc4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.535125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.536433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.538366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.538916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.548631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733db719-43e3-43d5-870e-96c6ad8e90f7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.554387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.556870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:19.580780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:19.581694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:19.582528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:19.582528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:19.582528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:19.582528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:19.589675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 272/539] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:19.591504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.592975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.593810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.594189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.594474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.606338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:19.611167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:19.635755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:19.636977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:19.637791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:19.639034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 268/540] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:19.643552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.686926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:19.715518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3cf7b-f7d8-4721-9fa1-4b657d4cf1f9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.763550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.764716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.765249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.765736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.766705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:19.767421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.769241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.795858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 273/541] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:19.798189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.846959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba5774-39cd-4eec-8732-4d204e15602c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.919129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07254039-4082-44fd-afc5-8ca143ea3311 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.925685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.926870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.927428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.928085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.929598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:19.930314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.933130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.950259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 269/542] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:19.957614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.958867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.959793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.960065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.960433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.962544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:19.968762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.970421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:19.987568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:19.988554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:19.989612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:19.995025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 274/543] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:19.996413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.070969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:20.124279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826a2af-ea4e-4dc1-b333-3ff2a0e67e4f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.196669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.197887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.205874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.206709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.232864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.234726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:20.267968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:20.269023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:20.269858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:20.269858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:20.269858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:20.269858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:20.273440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 270/544] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:20.274708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.279291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b93720-bf50-4d22-89b0-6491c71b6d22 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.337321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.338469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.338989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.339650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.341629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:20.342816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.345431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.384170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 275/545] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:20.391779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.511837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcc2f7-c3c4-4741-aee0-1e76c1391e38 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.545950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:20.552074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:20.598646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.599562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.600034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.600412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.601438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:20.601861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.603665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.635413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 271/546] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:20.641234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.658858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cb803-90e2-49a0-ac81-7dfd61854124 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.756054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.758149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.758671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.759189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.764723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=65d7683f531a41b1adf3ceb94d32ce2c) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:20.765213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.767634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27c104-62c5-4827-aa40-f1d64eeb70ca None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.801144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 276/547] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/role_assignments?user.id=65d7683f531a41b1adf3ceb94d32ce2c&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:20.829148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb26a9-c9da-435b-a21b-31d04779f23d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.829713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb26a9-c9da-435b-a21b-31d04779f23d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.829952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb26a9-c9da-435b-a21b-31d04779f23d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.832055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 277/548] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:29:20.838563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.838905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.839133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.879294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:20.884548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:20.930073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341cf6e-3e41-4868-8de8-aa2979d9ec6e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.987093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.994093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.994486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.994633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.006800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.008682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:21.054893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:21.056344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:21.057774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:21.061438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 272/549] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:21.069989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.075583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 278/550] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 03:29:21.113815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:21.117404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.211926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:21.265068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d556656-7599-47f5-8c01-dea5ee19cbfd None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.311335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.312556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.313267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.313758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.316439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ead44-60d9-4b05-b83e-a7f5776b805e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.337326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:21.346275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:21.347294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:21.348121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:21.352226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 279/551] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:21.361663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.425332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.426336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.426806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.427267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.429158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:21.431470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.433453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.473366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 273/552] 10.222.0.58 () {56 vars in 1135 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:21.481425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.560381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa300231-b791-47b7-920d-b132a61e3e5c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.577383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:21.609939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.611104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.611717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.612201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.613449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:21.613975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.616911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a74dd-1c02-44c9-8ce4-68009a4d31e6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.630355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 280/553] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:21.633334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b9248-e635-4b13-ab2e-40bb489322a0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.633912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b9248-e635-4b13-ab2e-40bb489322a0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.634936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b9248-e635-4b13-ab2e-40bb489322a0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.637401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 281/554] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:21.638394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b9248-e635-4b13-ab2e-40bb489322a0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.657312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:21.674535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041894f-1325-4605-9e16-0e5cf4be7b16 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.719131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.719787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.720126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.720410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.842002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b9248-e635-4b13-ab2e-40bb489322a0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.847897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.851271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.911395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.913394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.913886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.914350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.934703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.937465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-a16f65bd-2eb7-4bc0-b361-e3a124793dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:21.974128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:21.975129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:21.976119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:21.976119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:21.976119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:21.976119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:21.976794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 282/555] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:21.979031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.979535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.980036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.009592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:22.011490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:22.025921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.046356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 274/556] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:22.048445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.054409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.127280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 283/557] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:22.128251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.243196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1028121-fce9-4453-9ad8-6c75af351583 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.256778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff462a7-fcda-484e-b128-05f49386a5ce None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.294352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.295386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.295814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.296229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.306530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.307663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.308152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.308252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.308637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.309654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:22.309939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:22.310489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.313109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92f352-4f5d-4de9-b6f8-564a2d1dc043 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:22.328865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:22.330525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:22.331427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:22.331427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:22.331427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:22.331427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:22.331427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 284/558] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:22.331427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.335510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.342513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 275/559] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:22.343802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714080d-3c13-4b87-bc67-29fefe237122 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.344088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714080d-3c13-4b87-bc67-29fefe237122 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.344366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714080d-3c13-4b87-bc67-29fefe237122 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.346422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 276/560] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:22.347075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714080d-3c13-4b87-bc67-29fefe237122 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.429694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.478125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e4445-75ed-4c05-b4e2-ffc1b6f32190 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.492728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714080d-3c13-4b87-bc67-29fefe237122 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.522815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.524122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.524623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.525072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.540868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:22.544974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.546018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.546487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.546921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.548154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:22.548691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.551090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c894c6-b89c-4a7b-a303-80f6a417d2de None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:22.555373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:22.556323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:22.557511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:22.560877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 285/561] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:22.562079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.572716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 277/562] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:22.575224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.575795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.576292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.599072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:22.600825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:22.638215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.685440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 278/563] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:22.691074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ce48-5acb-41ed-be6e-7e67f5164a19 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.693479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.738775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.739581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.739861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.741933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.750028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.750430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.751495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:22.770668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:22.772734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:29:22.774071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:22.776955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 286/564] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:22.777854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.824010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e81949-a6c1-4903-9ef2-4cc6772801bb None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.870333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.871022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.871307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.873921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.874580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:22.874891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.876310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:22.886974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 279/565] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:22.888129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.922388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee902d29-6dbb-4ec9-a19c-0eb72e9e19c2 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.962647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:22.979418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.980766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.981152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.981613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.991300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.993616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:23.012308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:23.014117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:23.015912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:23.018346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 287/566] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:23.019324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.038710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2e3ee-2fb1-4204-9a48-45256a2b563f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.042327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.083925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.085239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.085709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.086097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.100286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:23.109763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:23.111115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:23.112226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 280/567] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:23.113632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.174590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06967ee-a28c-4072-9ad8-5adf0733a63a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.230520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.231380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.231955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.232380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.233318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:23.233657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.235746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.269899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 288/568] 10.222.0.58 () {56 vars in 1137 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:23.275634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e720f-badb-4566-bf3d-5ad9e4469022 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.276392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.327289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.328127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.328411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.328745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.341419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.343087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.345927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:23.365036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:23.366875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:23.368529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:23.368529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:23.368529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:23.368529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:23.368529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 281/569] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:23.370087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.391359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.420389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662725c3-244b-45a1-91d1-c7bc6a7a8473 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.457673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.458339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.458626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.459459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.460263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:23.460650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.462092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.477079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 289/570] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:23.478320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.492096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5cac-1d94-44a6-99f2-520517fa2e82 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.540303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.541406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.541869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.542321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.543434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:23.543944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.546372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1edc5-d184-4d3c-9444-4c4def5bc52e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.558472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 282/571] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:23.560801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2d033-b8ab-4ac0-b3b9-acf1c1794b0c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.561301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2d033-b8ab-4ac0-b3b9-acf1c1794b0c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.561795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2d033-b8ab-4ac0-b3b9-acf1c1794b0c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.564540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 283/572] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:23.565474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2d033-b8ab-4ac0-b3b9-acf1c1794b0c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.606170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab13cd-7e2b-4c90-bfcd-289ec6b87a95 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.608124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.649653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.650460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.650743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.651017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.661630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.663304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.664768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:23.683848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:23.685059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:23.686150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:23.686150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:23.686150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:23.686150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:23.686150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 290/573] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:23.686150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.689302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2d033-b8ab-4ac0-b3b9-acf1c1794b0c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.739664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.741398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.741916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.742502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.743790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:23.744333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.746857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.765774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 284/574] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:23.766643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.797515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50cf2a-a5ca-4081-a5f8-283d364a01ed None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.839463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.840131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.840417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.840734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.841568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=1c8eaac98b1b4f5db6c0b60c3f01f711) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:23.841889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.843340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b559-52b2-43bb-8111-fa18e55f3fff None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:23.849167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 291/575] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/role_assignments?user.id=1c8eaac98b1b4f5db6c0b60c3f01f711&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:23.850411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.850821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.851104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.869354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:23.870724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:23.892815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.904941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f2be0-6cb6-40c9-994d-60df73fa97d4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:23.957789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 292/576] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:23.958741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:23.975364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:23.976708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:23.977277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:23.977816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:23.987312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:23.997789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:23.997789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:24.015656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:24.016651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:24.017530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:24.017530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:24.017530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:24.017530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:24.019189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 285/577] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:24.020222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.114690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7858c10-b910-455e-a472-b495b5e13a87 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.154288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae61966-0037-4645-86b3-3702a6af9201 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.168474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.169702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.170215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.170690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.172093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:24.172669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.175362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.203238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 293/578] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:24.203238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.209171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.210035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.210343 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.210617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.220301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.221835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.226570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:24.243437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:24.246235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:24.248957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:24.251329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 286/579] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:24.251329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.325815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.385810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0b8f3-6375-49e9-8e17-a23c4a149946 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.386109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5fbd1-2406-4741-834e-1776c51f9a5a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.430571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.431535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.431535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.432295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.432613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:24.432921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.434342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.439437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.440774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.441221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.442319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.458213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 287/580] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:24.458213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.458488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:24.466387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:24.468023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:24.469109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:24.472640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 294/581] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:24.473446 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.567321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.609808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9683b9-6ecd-4060-80e4-9bfca093d79d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.635653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819a61c-23b8-4ed2-9bac-659a0e82a4c7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.648898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.666878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.667981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.668475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.668984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.670629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:24.675677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.675677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.681966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.682872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.683225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.683573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.691516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.695322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.716391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 295/582] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:24.716391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:24.726958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:24.736751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 288/583] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:24.738742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.861702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b36e0-c2df-4e8b-bfea-939314919f48 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.871359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20028-0774-4154-85b4-9850b55ac598 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.902323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:24.910248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.911073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.911448 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.912311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.913172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:24.913540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.914993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.917839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:24.919213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.919700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.920181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.925326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 289/584] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:24.926313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.937738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:24.942113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:24.959900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:24.961158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:24.962106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 296/585] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:24.962704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.978181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:25.077114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8c7b2-19c2-449d-b246-210a06e272d6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.086944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed1ed-eff7-4c89-a2c4-1e7b555a4b9d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.126391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.127357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.127647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.127962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.139572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.141098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.141772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.143360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.144257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.144972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.146980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:25.147516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.150238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:25.161619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:25.164373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:25.167392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:25.167392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:25.167392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:25.167392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:25.167392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 290/586] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:25.167392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.181851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 297/587] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:25.181851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.242414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:25.270562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:25.293477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92b9e8-23d5-47d3-b087-678c4bdaf054 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.319135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aeb7a8-a760-4217-8818-0c3e4bc74ccd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.355374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.355825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.356559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.356814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.356916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.357189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.357304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.357750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.357925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:25.358235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.359679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.379112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.382023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.382267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 298/588] 10.222.0.58 () {56 vars in 1137 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:25.383045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:25.412367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:25.414385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:25.424264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:25.424264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:25.424264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:25.424264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:25.424264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 291/589] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:25.424264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.533534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcfdb5-53e2-42e7-ae85-b500c1b44741 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.537805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:25.538810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8be91-6e02-4251-9356-d47be237c8f5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.584418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.585387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.585663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.586136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/dc6f327db350476e9e5194719da4cbb5/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.590836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.591365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:25.591749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.592600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.592719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.593769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:25.594212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.596415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.615379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=dc6f327db350476e9e5194719da4cbb5, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.617594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.624919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 299/590] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:25.624919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.708541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 292/591] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:25 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/dc6f327db350476e9e5194719da4cbb5/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:25.708541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.772181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162175da-deea-4f6c-950a-009fd45e52dc None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.839805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.841372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.841973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.842562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.863211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.866646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.871272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e3d97-8a26-4b80-ad95-5d8295d0061d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:25.904783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:25.906002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:25.907034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:25.907034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:25.907034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:25.907034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:25.913126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 300/592] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:25.914669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:25.939606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.940629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.941083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.941500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.942397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:25.942822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:25.944118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.945868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.959505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 293/593] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:25.960650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.022962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:26.057850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fb492-4147-4adf-8daa-8d3d68b97030 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.104692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.105716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4863-6e73-4de4-a872-2591addc6174 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.105998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.106432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.106853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.117333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.119151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:26.143061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:26.144169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:26.145252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:26.154125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 301/594] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:26.155347 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.182664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.184058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.184726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.185362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.187054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:26.187735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.190952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.221196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 294/595] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:26.223094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.251596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:26.293159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dfa82-8b7e-49b9-b9e2-9d0c83ca018b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.347969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.348841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.349606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.349921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.359083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.360596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:26.378705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:26.380357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 302/596] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:26.383381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.420743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de551e2b-6407-4160-ad97-c3ebc678d163 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.430450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:26.460761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.461436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.461726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.462004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.462806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:26.463117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.464610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70160cd-87c5-42e9-9b62-4391ffbbf0c6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.479870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 295/597] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:26.481521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399777f2-874c-423c-b389-22d561c92ceb None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.481814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399777f2-874c-423c-b389-22d561c92ceb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.482096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399777f2-874c-423c-b389-22d561c92ceb None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.483583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 296/598] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:26.488302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399777f2-874c-423c-b389-22d561c92ceb None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.514346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7b83-0a98-463f-888f-98e2d2106372 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.553832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.554633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.554917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.555196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.558224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:26.562885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.569165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:26.580184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:26.581762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 303/599] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:26.583780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.627138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399777f2-874c-423c-b389-22d561c92ceb None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.685798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.686918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.687464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.687957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.689112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:26.689625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.691452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c29d0-d3b2-417d-a612-7568441e0c32 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.707998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895855c-128d-440f-87e3-acd8639e0b5f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:26.718804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 297/600] 10.222.0.58 () {56 vars in 1139 bytes} [Fri May 27 03:29:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:26.718804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.718804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.718804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.734649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:26.735778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:26.749649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:26.750573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:26.750857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:26.751131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/0ceb759c7b4b4292bb9763574d79254b/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:26.757686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:26.767873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=0ceb759c7b4b4292bb9763574d79254b, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:26.769494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:26.817651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 298/601] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:26.818680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.848541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 304/602] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:26 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/0ceb759c7b4b4292bb9763574d79254b/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:26.848541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:26.939733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:26.995775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bee09-82c0-493e-a3ee-c6872ebc9b86 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.010254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb7b6f-6f43-4136-9194-d3a6692c764e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.050960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.051787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.052076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.053233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.057282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.058218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.058218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.058513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.059935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:27.059935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.061897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.071643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.074366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.076806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 299/603] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:27.077678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:27.095702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:27.097738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:27.098621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:27.098621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:27.098621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:27.098621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:27.098621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 305/604] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:27.098621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.108065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:27.231016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:27.237128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2162725-eb05-41ed-842c-3e8b4b5c8786 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.262443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e619-64f6-4187-94f2-c6fae89bc63b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.293228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.294425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.294711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.294990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/cd4fb121b8de406899d5698c84d0903b/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.305714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.306990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.307535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.308028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.316528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=cd4fb121b8de406899d5698c84d0903b, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.318593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.321464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:27.330114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:27.331199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:27.332068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:27.336587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 300/605] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:27.338011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.425094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 306/606] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:27 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/cd4fb121b8de406899d5698c84d0903b/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:27.427042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.464194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:27.552054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76158288-1b35-4470-97fc-656ac9462b9d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.602520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c22e51-9d74-405e-bccf-34c7fca22721 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.638949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.639702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.640012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.640294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.641183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:27.641499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.642930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.648829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.649881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.650380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.650863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.651838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:27.652398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.654899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.666521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 307/607] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:27.675561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 301/608] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:27.677845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.680744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.693693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:27.803039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beb745-b717-47b0-bfc9-b9aba819c180 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.844309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef636a0c-5716-4b1a-98fb-db6d99be9825 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.846973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.848078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.848583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.849097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.860675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.862506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:27.883175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:27.884158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:27.885060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:27.885060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:27.885060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:27.885060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:27.885060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 308/609] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:27.893598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.948307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.949162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.949454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.949717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.953796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:27.968103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.969660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:27.999623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:28.003118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:28.006241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 302/610] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:28.012534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:28.016705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:28.100767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a479b-0a54-48a4-b9cc-5f28286b230e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:28.167773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:28.168451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.168767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.169022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.169757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:28.170066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:28.171454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:28.185957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eb19e-0848-402b-a1ed-d7635fe66bbe None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:28.198002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 309/611] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:28.209629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:28.260019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:28.260726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.261016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.261297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.262100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:28.264855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:28.268335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:28.294711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 303/612] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:28.304838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:28.358734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda52aaf-08f4-42a8-8e5c-7c404c36687e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:28.385717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:28.399000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:28.399814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.400100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.400368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.413114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:28.416255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:28.436817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:28.443103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:28.447324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:28.448676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:28.451883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 310/613] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:28.454474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:28.527032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14043-6e37-4768-8a47-dac581ef0658 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:28.590167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647fe2e-1386-4361-89a2-dccb86a3428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:28.594180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647fe2e-1386-4361-89a2-dccb86a3428f None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.594958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647fe2e-1386-4361-89a2-dccb86a3428f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.595458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647fe2e-1386-4361-89a2-dccb86a3428f None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/65d7683f531a41b1adf3ceb94d32ce2c/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.611599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be796a0-9919-4989-88e0-f93da9a4c561 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:28.633929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647fe2e-1386-4361-89a2-dccb86a3428f None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=65d7683f531a41b1adf3ceb94d32ce2c, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:28.638337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647fe2e-1386-4361-89a2-dccb86a3428f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:28.661068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:28.663188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.664024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.664860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.666765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:28.667601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:28.670479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34b16b-5642-4f9c-b7a9-2e191e8bd9be None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:28.696837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 311/614] 10.222.0.58 () {56 vars in 1135 bytes} [Fri May 27 03:29:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:28.698868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73450-3118-4f0a-9e0f-d0160e6eebab None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.699429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73450-3118-4f0a-9e0f-d0160e6eebab None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.700010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73450-3118-4f0a-9e0f-d0160e6eebab None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.702299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 312/615] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:28.703511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73450-3118-4f0a-9e0f-d0160e6eebab None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:28.714397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 304/616] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:28 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/65d7683f531a41b1adf3ceb94d32ce2c/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:28.715702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.715985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.716259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.764410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:28.768148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:28.822517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:28.906577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73450-3118-4f0a-9e0f-d0160e6eebab None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:28.948771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:28.949594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:28.949879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:28.950151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:28.962411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 305/617] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:28.969862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:28.971662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:28.992567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:28.993570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:28.994449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:28.994449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:28.994449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:28.994449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:29.000104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 313/618] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:29.002938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.041739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.063384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:29.180805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00f70b-1f2c-4a4e-9140-7721eaf20b88 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:29.210959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:29.220841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bc99b-de70-4857-adfe-eb15c92f1dd6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:29.245829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:29.246524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:29.246813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:29.247093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:29.247839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:29.248154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:29.249605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:29.276877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 306/619] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:29.284372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.285401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:29.286214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:29.287021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:29.287303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:29.325267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:29.336692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:29.337681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:29.338525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:29.341861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 314/620] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:29.350412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.464211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:29.474753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ee43f-7815-4d40-938e-bdd756f161ed None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:29.526679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:29.527504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:29.527793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:29.528075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:29.530397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d453e12-a2b5-477a-bc81-eb89539300f7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:29.545258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:29.547178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:29.571106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:29.572103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:29.573021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 307/621] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:29.582715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.597487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:29.598563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:29.599134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:29.599623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:29.600770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:29.601308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:29.603056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:29.613498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 315/622] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:29.625841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.661271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:29.754019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6274ca-8da4-4660-bb13-6db4d2100970 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:29.768535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e13a31-5348-4032-9c27-7e298239190a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:29.814115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:29.814932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:29.815218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:29.815487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:29.831295 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:29.832887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:29.855928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:29.856933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:29.858375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:29.858375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:29.858375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:29.858375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:29.858375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 316/623] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:29.867024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.881792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:29.886048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:29.886387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:29.886670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:29.889423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:29.889738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:29.894276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:29.928452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 308/624] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:29.949548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:29.977479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:30.091324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:30.110959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87106d7f-fda1-4a87-8b3b-c7b7be103fa5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.140740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c920f50-bea1-4cf8-85e5-0c0792429581 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.152109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.153095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.153376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.153650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.154385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:30.155012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:30.156093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5366f-99a2-4b1f-9fdb-5847a3970fcc None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:30.172685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 317/625] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:30.174252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4b4b6-b2d8-458e-b82d-00374a4f16bc None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.174608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4b4b6-b2d8-458e-b82d-00374a4f16bc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.175144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4b4b6-b2d8-458e-b82d-00374a4f16bc None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.176438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 318/626] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:30.177920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89219f-b36d-413f-8ce4-0048dc09690e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.178258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89219f-b36d-413f-8ce4-0048dc09690e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.178620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89219f-b36d-413f-8ce4-0048dc09690e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.181627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 319/627] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:30.181627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89219f-b36d-413f-8ce4-0048dc09690e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.195826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80a1c8-b506-49b9-ac15-a345a194843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.196779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80a1c8-b506-49b9-ac15-a345a194843b None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.197142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80a1c8-b506-49b9-ac15-a345a194843b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.197584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80a1c8-b506-49b9-ac15-a345a194843b None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/1c8eaac98b1b4f5db6c0b60c3f01f711/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.215522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80a1c8-b506-49b9-ac15-a345a194843b None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=1c8eaac98b1b4f5db6c0b60c3f01f711, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:30.217012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80a1c8-b506-49b9-ac15-a345a194843b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:30.296545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 309/628] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:29 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/1c8eaac98b1b4f5db6c0b60c3f01f711/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:30.296545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112fa2-f377-4d85-9bb5-6ddbd7bd43ef None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.296545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112fa2-f377-4d85-9bb5-6ddbd7bd43ef None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.296545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112fa2-f377-4d85-9bb5-6ddbd7bd43ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.324278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-cb112fa2-f377-4d85-9bb5-6ddbd7bd43ef None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:30.325583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-cb112fa2-f377-4d85-9bb5-6ddbd7bd43ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:30.341103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89219f-b36d-413f-8ce4-0048dc09690e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.397590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:30.411012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.412607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.413126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.413633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.420912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:30.429615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:30.431869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 310/629] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:30.433143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:30.433261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.433511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.433860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.454953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:30.461165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:30.465987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:30.469259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 320/630] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:30.470700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.570911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 311/631] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:30.571805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.647476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efad75-85b9-48f2-8d8d-ff2056d7ae3a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.703478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20985f28-1faf-48e1-8720-c2624aec5bb2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.709360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.710372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.710783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.711130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.731075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:30.734778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:30.743811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.744929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.745326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.745675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.746657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:30.747034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:30.748685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:30.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:30.750979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:30.752065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:30.753431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 321/632] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:30.758598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.774262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 312/633] 10.222.0.58 () {56 vars in 1137 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:30.775196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.820350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:30.899180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3dc3-301b-4ca9-921a-4176e781f08b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.922366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a03a9-de1e-4329-93e3-3abc80773bbc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.944913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.946878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.947624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.948269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.962469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:30.973048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:30.974582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:30.975435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:30.983545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 322/634] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:30.983545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.990582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.991765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.992206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.992925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.994187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:30.994600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:30.997137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.009012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 313/635] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:31.010123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.088766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.115607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf6ac7-c2a3-41ad-80d5-8de18fcacec9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.127229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.162768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.163760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.164118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.164780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.167156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9837c2-b4fd-4685-8419-2f47bf20a3f7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.210627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.212040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.212927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.213433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.214609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:31.215127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.217392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.232817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 314/636] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:31.233637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.274263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.276348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.362496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264156d-289d-4ee9-a023-3cc97acf7981 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.371134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.399051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.399849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.400134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.400412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.413227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.414688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.417974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 323/637] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:31.419135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.429185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:31.433969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:31.434856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:31.435734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:31.435734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:31.435734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:31.435734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:31.435734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 315/638] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:31.436489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.535947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94740d5-1431-4929-8a6d-5ee1640a2bd8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.550998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589d4fa-ecc4-4574-809c-31f12cd2197c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.574460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.575389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.575811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.576092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.589087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.591058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.591587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.592656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.593122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.593572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.606992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.608868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:31.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:31.612772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:31.618445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:31.618445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:31.618445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:31.618445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:31.618445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 324/639] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:31.618748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:31.633249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:31.634375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:31.635262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:31.635262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:31.635262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:31.635262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:31.637221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 316/640] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:31.638245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.652561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.685473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.734900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d7316-d11b-429e-aed6-b8ff843b87c2 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.757209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81426b4b-54cc-42b1-afd6-0bc32c988ac2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.773668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.774385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.774658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.774934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.775669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:31.776066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.777532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.791290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 325/641] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:31.792453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.799972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.800553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.800875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.801228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.802555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:31.803594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.805172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65384fea-c55f-46b0-b3f7-28eb7df32f5f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.820221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 317/642] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:31.825052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b277-77a6-4c7d-bf5d-7db410900190 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.825052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b277-77a6-4c7d-bf5d-7db410900190 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.825052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25b277-77a6-4c7d-bf5d-7db410900190 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.825052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 318/643] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:31.825052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25b277-77a6-4c7d-bf5d-7db410900190 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.906561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.932130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ea4f6-aeb9-4513-b3e4-fcb16238bd3e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.939168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25b277-77a6-4c7d-bf5d-7db410900190 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:31.953211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.973335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.974579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.974859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.975342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.976148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:31.976465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.977851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:31.977985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.978658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:31.978950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:31.979502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:31.987589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:31.989035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:31.992295 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 326/644] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:31.993385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-8558a017-3cac-470c-980d-6deb4780ad9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:32.012138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:32.013365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:32.014664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:32.016725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 319/645] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:32.018217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.018509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.018789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.038706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:32.040145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:32.112589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbe329-c8ae-4b6c-9985-630db4b04c8e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.157520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 320/646] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:32.158773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.165905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.167688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.168264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.168842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.187374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.189041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:32.205706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:32.207497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:29:32.209954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:32.212008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 327/647] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:32.212868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.259115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.262911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.329877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d60d-94ec-4c1f-a057-30a95ea7273e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.352041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d2005-05f8-458f-8714-941f190e16bd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.379858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.381459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.381742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.382147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.393994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.398565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.401970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.403118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.409199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.409199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.409199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:32.409199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.409199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:32.424332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:32.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:32.426606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:32.428286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 321/648] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:32.429314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.438421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 328/649] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:32.439761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.564526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15d263-dee3-4a02-a3b2-a4ca99c31962 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.571933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.578687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f643b-71c4-4ce1-afdf-cedfbcce9f03 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.588443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.604244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.605237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.605357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.605634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.617024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:32.618557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.619293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.619596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.619862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.620757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:32.621083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.622519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:32.624794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:32.626371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:32.627204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:32.631661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 322/650] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:32.633103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.651113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 329/651] 10.222.0.58 () {56 vars in 1137 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:32.651113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.754086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5add4be-5163-42eb-97af-082ab5ff7fc6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.788130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744c554-afe0-4b75-bff7-a9ac2fec75c5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.791529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.792201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.792489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.792783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.793587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:32.793889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.795331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.807284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 323/652] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:32.808319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.826296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773ba2c-539d-4d39-86ee-55c1c8c2f11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.826957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773ba2c-539d-4d39-86ee-55c1c8c2f11a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.827242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773ba2c-539d-4d39-86ee-55c1c8c2f11a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.827523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773ba2c-539d-4d39-86ee-55c1c8c2f11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.851270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.853320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.903788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773ba2c-539d-4d39-86ee-55c1c8c2f11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.903788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773ba2c-539d-4d39-86ee-55c1c8c2f11a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.915641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc90cae-4e95-4338-92f1-f0a50551273e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.954140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.954816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.955107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.955385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.956169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:32.956469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.957881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.967788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 324/653] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:32.968673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.010127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 330/654] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:33.011516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c5298-46ff-4503-a6a5-e306d02b7164 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.011863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c5298-46ff-4503-a6a5-e306d02b7164 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.012152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c5298-46ff-4503-a6a5-e306d02b7164 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.013710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 331/655] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:33.016613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c5298-46ff-4503-a6a5-e306d02b7164 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.076752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.088961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca2a8d-d675-4097-8355-872d8ec51887 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.090501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.123460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c5298-46ff-4503-a6a5-e306d02b7164 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.124445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.125131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.125394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.125673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.160717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.161325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.161607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.161886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.162704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=dc6f327db350476e9e5194719da4cbb5) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:33.163014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.164455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.170479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 332/656] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/role_assignments?user.id=dc6f327db350476e9e5194719da4cbb5&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:33.171273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.221254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.222761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.310817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdd436-09dd-4bc0-b109-bdb205c4933c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.336340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.359336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.360196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.360486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.360798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.372567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 325/657] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:33.372567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.372567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.379601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.384401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-0f345059-4be8-4e20-be55-7d766394f73d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:33.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:33.402905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:33.404364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:33.404364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:33.404364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:33.404364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:33.404364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 333/658] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:33.405381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.405541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.405798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.425368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:33.426610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:33.513863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 334/659] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:33.515056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.528750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8640bc-d5d6-42a6-a6b7-6fdeaf44fab6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.571169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.571980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.572318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.572728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.584263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.585814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:33.607412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:33.608482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:33.609432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:33.609432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:33.609432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:33.609432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:33.612967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 326/660] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:33.614586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.640472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.644848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bce15d-645e-47a9-8068-1573e81997c7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.683584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.685247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.686053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.686334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.686605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.706628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.708192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:33.728938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:33.730079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:33.731175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:33.731175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:33.731175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:33.731175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:33.732926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 335/661] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:33.733806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.747601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260610-e3f3-49f9-9bf3-aeb0e0e750e9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.793258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.794077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.794459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.794841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.796331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:33.796846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.798463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.812764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 327/662] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:33.813617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.861484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e36d3-775f-4f21-8edd-9edc02109fe2 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.910466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.912464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.913196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.913781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.926196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:33.936249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:33.937257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:33.938100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:33.941108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 336/663] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:33.942240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.954186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18903b-6d43-4a32-b862-e7aa56941bed None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.963207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.973280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.002616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.003296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.003583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.003861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.004793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.005645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.007596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.023901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 328/664] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.024981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.076529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35206947-12a7-4407-a720-297dac5a5fc1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.119656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.120541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.120675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.120897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.121651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.121947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.123350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.141300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 337/665] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.142255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.143122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd783f0-ae47-4882-8cef-631061d6143b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.193982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.194873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.195274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.195648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.204112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.205912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:34.221378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:34.222561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:29:34.223854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:34.230215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 329/666] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:34.230215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.241270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.268389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.281932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48497c7d-8768-4011-ba1e-0edce6235e26 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.318281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.318973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.319351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.319631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.320378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.320724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.322105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.330555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 338/667] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.331367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.347602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de439f6-8256-48fb-a982-2f3f8947dee6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.387100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.388015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.388300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.388626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.396366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.398205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:34.413497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:34.414432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 330/668] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:34.415932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.448214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52d78d-a63b-4a5e-9044-6ae730c10250 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.486947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.487746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.488026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.488297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.491666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.497992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.499520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:34.516902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:34.523382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 339/669] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:34.524287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.530706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.538067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274f8a7-08c6-41cd-b6d8-ec0ba23767cc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.578190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.578882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.579166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.579442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.580323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.581068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.582130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.639456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafd40c-1a5e-4829-987f-e6424c6d98c7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.680591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.681412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.681690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.681964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.694032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.695536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:34.714516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:34.715460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:34.716455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:34.716455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:34.716455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:34.716455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:34.716455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 340/670] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:34.717271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.729624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 331/671] 10.222.0.58 () {56 vars in 1139 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.731553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.773490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.830379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d1003-68fc-49c5-bfdf-42428f7feefe None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.867627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7914772-26ea-495e-9f0e-30b052c6f6e7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.893376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.894079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.894308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.894577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.895370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.896262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.897761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.910916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.912242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 341/672] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.913057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.919920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.920324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.920733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.921815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.922568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.924078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.938405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 332/673] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.939459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.962423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.044813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b5718-9ed6-4ed2-aacd-c521d2e84a3c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.057372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.060327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e9023-6c16-4866-978f-6a1cd99242de None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.095460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.096144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.096429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.096748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.097468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:35.097781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.099383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.106537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.107415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.108406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.108857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.113011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 342/674] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:35.113868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.206363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.208063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.223452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.232695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3161-f2df-4d48-a750-4ac4b3526a78 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.275792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.276442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.276742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.277018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.277843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=0ceb759c7b4b4292bb9763574d79254b) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:35.278152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.279593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.287334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 343/675] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/role_assignments?user.id=0ceb759c7b4b4292bb9763574d79254b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:35.288081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.324547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.339361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 333/676] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:35.341187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.420633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dcb6d-20e3-4b13-b0d7-27cf3774a0be None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.465487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.466495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.466495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.466779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.467571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=cd4fb121b8de406899d5698c84d0903b) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:35.467880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.469426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.476968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 344/677] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/role_assignments?user.id=cd4fb121b8de406899d5698c84d0903b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:35.477974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.514347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a257b3-a40b-4adf-b160-e232191a3237 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.536523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.566642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.568891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.569394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.569866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.580398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.582201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:35.600731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:35.601744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:35.602682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:35.605450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 334/678] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:35.613744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.659864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.668103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac119-7189-445d-b2ab-3f09e936ff0e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.716166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.717367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.717737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.717910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.733325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.734303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-330a27a7-84ca-4a67-9248-88dea3faaca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:35.761083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:35.762070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:35.762965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:35.762965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:35.762965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:35.762965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:35.762965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 345/679] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:35.764316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849af5c-e1a0-4e94-9e54-a03e4c435436 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.764843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849af5c-e1a0-4e94-9e54-a03e4c435436 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.765367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849af5c-e1a0-4e94-9e54-a03e4c435436 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.767749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 346/680] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:35.772732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849af5c-e1a0-4e94-9e54-a03e4c435436 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.777906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78075a-cc2e-4bc4-a264-608eea1b96a1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.826731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.827068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.827362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.827700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.829141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:35.829141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.830805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e8fe3-dcb7-4869-b174-d0fd851d2fae None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.853834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 335/681] 10.222.0.58 () {56 vars in 1135 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:35.860493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.862387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.862387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.883325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:35.885095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:35.910055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849af5c-e1a0-4e94-9e54-a03e4c435436 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.925218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.954289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.955145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.955558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.956551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.958118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:35.958990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.960702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.975794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 347/682] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:35.977092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.983730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 336/683] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:35.984845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.996934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.100191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b248-a5b5-45f5-bcb2-03a207576c01 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.137482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dd49e-78cb-4a9d-96e4-95a5dbde5abb None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.145149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.146732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.147116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.147496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.184644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.185689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.186118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.186563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.194827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.196561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:36.212095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:36.213087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:36.213972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:36.216535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 337/684] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:36.217681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.231266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.232964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.243650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.246843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.346126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33665dc7-26bb-4b62-9a3e-17b6587a7c4b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.370195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 348/685] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:36.371505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.391209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.392044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.392330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.392660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.405102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.406747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:36.427848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:36.428846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:36.429729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:36.429729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:36.429729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:36.429729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:36.429729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 338/686] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:36.430425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.501487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d96241-093d-48e0-a95c-745bbf511250 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.514129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.554660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27d96d-c62f-4d57-b379-f4971b34a9f4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.598288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.599162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.599449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.599728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.612933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.614960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:36.635764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:36.636723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:36.637598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:36.637598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:36.637598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:36.637598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:36.641794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 339/687] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:36.642865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.649700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.681053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.682946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.683501 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.683971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.684990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:36.685498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.687170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.701187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 349/688] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:36.702382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.754883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464df01-1c21-4fe0-82db-9011f7a1fde5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.780325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.793201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.805730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.805730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.805730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.805730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:36.805730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.805730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.810864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 340/689] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:36.811843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.825974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4599cdd5-d24f-4ff8-a243-47156ee35153 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.881780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.882484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.882765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.883039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.883774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:36.884078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.885493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:36.899961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 350/690] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:36.899961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:36.934667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:36.945371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837450c-eb47-4bfb-bdd6-01f69912364b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:36.992151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:36.993120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:36.993502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:36.993896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:36.995188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=65d7683f531a41b1adf3ceb94d32ce2c) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:36.995709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:36.998665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:37.012536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 341/691] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:36 2022] GET /identity/v3/role_assignments?user.id=65d7683f531a41b1adf3ceb94d32ce2c&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:37.012536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:37.031416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df369728-f01c-4363-b770-4e804a80fad7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:37.071829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:37.106339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee95329-e8e6-4b4e-8442-ec8526219325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:37.107371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95329-e8e6-4b4e-8442-ec8526219325 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:37.107843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95329-e8e6-4b4e-8442-ec8526219325 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:37.108308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee95329-e8e6-4b4e-8442-ec8526219325 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:37.109228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee95329-e8e6-4b4e-8442-ec8526219325 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:37.110975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee95329-e8e6-4b4e-8442-ec8526219325 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:37.132525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d6897-2b87-43f2-9921-1714a53270d4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:37.172234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:37.173207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:37.173589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:37.173875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:37.182488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:37.183969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:37.200063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:37.201213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:37.202310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:37.206466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 342/692] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:37.208420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:37.327033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecae14a-79a8-4673-8ab7-a688372db490 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:37.358258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:37.372297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:37.372297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:37.372297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:37.372297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:37.373395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:37.373395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:37.374446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:37.389265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 343/693] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:37.396395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:37.425465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 351/694] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:29:36 2022] POST /identity/v3/users => generated 254 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:37.579398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dffab-fb4a-44dd-a264-c1301752d18c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:37.654964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:37.656110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:37.656661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:37.657176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:37.658889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=1c8eaac98b1b4f5db6c0b60c3f01f711) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:37.659469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:37.662172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73271b8-f921-4dae-bb20-0c605dc3c6ea None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:37.673360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 344/695] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:37 2022] GET /identity/v3/role_assignments?user.id=1c8eaac98b1b4f5db6c0b60c3f01f711&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:38.098542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33748f-b090-4461-8fe9-11cd0e1f289b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.099058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33748f-b090-4461-8fe9-11cd0e1f289b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.099372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33748f-b090-4461-8fe9-11cd0e1f289b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.101356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 352/696] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:38.107161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb486f-5558-434a-860e-1dfe17a41d92 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.107568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb486f-5558-434a-860e-1dfe17a41d92 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.108010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb486f-5558-434a-860e-1dfe17a41d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.135468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-aacb486f-5558-434a-860e-1dfe17a41d92 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:38.137709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-aacb486f-5558-434a-860e-1dfe17a41d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.184450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:38.244821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 345/697] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:38.247594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee8ca-de17-401f-a38c-cf73e7d9f5b7 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.248059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee8ca-de17-401f-a38c-cf73e7d9f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.248533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ee8ca-de17-401f-a38c-cf73e7d9f5b7 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.251560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 353/698] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:38.257028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.257326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.257597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.271943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee8ca-de17-401f-a38c-cf73e7d9f5b7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.275740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:38.276964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:38.426427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 346/699] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:38.457601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.511269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:38.548895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ee8ca-de17-401f-a38c-cf73e7d9f5b7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.621490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.623573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.624413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.625263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.647670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:38.653995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:38.666301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:38.668153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:38.669839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:38.675739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 354/700] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:38.682832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.688492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb570a5-01a2-4d5c-beb4-e42cfd77aa05 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.750404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.751482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.751746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.752016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:38.752655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:38.754060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:38.784179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:38.787389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:38.789157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 347/701] 10.222.0.58 () {56 vars in 1129 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:38.791973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:38.921967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5dee4-f223-4c3b-bbd5-d2ad69bb2809 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.977293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea940962-36a7-4bcc-a125-4063214f3232 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:38.996124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:38.997916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:38.998854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:38.999620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.001571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:39.002411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.005818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.024077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 355/702] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:39.036537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.042556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:39.073369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.074294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.074478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.074619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.075373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:39.075686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.077209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.093029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 348/703] 10.222.0.58 () {56 vars in 1124 bytes} [Fri May 27 03:29:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:39.099216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.145115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:39.205866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db926b9b-6c1c-4a1d-b576-2340d1cabd37 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.249294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.250450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.251193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.251672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.264425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.266377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:39.286324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:39.287538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:39.288449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:39.288449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:39.288449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:39.288449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:39.290406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114f461-33fd-42b5-be8f-efb1e0731147 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.291164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 356/704] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:39.297919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.365350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.366074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.366376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.366650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.367415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:39.367679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.369127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.382742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 349/705] 10.222.0.58 () {56 vars in 1124 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:39.394579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:39.482191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5cea-1ac6-4fc1-ac28-36d462cfb061 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.530361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.531403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.531893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.532383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.533329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:39.533844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.535600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.551367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 357/706] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:39.561686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.702167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:39.719003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272e36-063b-4cfa-8b71-553556bbeee8 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:39.764935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:39.765782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.766064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.766351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.778768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:39.782350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-5289d07a-7e3c-4785-abfe-b20c970ffe9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:39.813139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:39.815027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:39.818960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 350/707] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:39.823127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:39.996538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0195-98af-4a49-be98-052282f1fa68 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:39.996538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0195-98af-4a49-be98-052282f1fa68 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:39.996538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0195-98af-4a49-be98-052282f1fa68 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:39.998014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 351/708] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:40.007180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.007465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.007794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.026684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:40.027465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:40.101272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61174088-c481-4ffa-90bd-c9c31d43b39a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.152021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.204151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 352/709] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:29:40.214501 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.216651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.217407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.218240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.220095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:40.220973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.227724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.266024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.281879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 358/710] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:29:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:40.289169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.425743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.454444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef8af-9708-47fe-96e7-d5bddb939aa9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.495541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.496394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.496714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.497057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.497800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.499247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.501322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18f927-97e3-424b-ab02-4ccd3f4f91b0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-549ccb0d-3bdc-43a4-b1c8-760037d8e9a1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:40.512361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:40.513445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:40.514636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 353/711] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:40.516804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e0a6-67d9-4294-8526-f11d22d87d21 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.517196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e0a6-67d9-4294-8526-f11d22d87d21 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.517576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e0a6-67d9-4294-8526-f11d22d87d21 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.523592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 354/712] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:40.524664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48e0a6-67d9-4294-8526-f11d22d87d21 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.558567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.559502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.559953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.560807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.618979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.653751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.655624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.694299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48e0a6-67d9-4294-8526-f11d22d87d21 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:40.749858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:40.751495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.751930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.752368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.753278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] RBAC: Adding query filter params (name=network) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:40.753745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:40.755960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f78256-29d6-4eea-8f76-3c8913164c19 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:40.763649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 355/713] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:40.765493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:40.766220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:40.766666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:40.774559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.791768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:40.793492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:40.799588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 359/714] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:29:40.800702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.906993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 356/715] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:40.908275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:40.923029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.970348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ab66-fff0-4c42-a5fe-11341dd5dec5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.014657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.015362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.015651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.015928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.016682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] RBAC: Adding query filter params (type=network) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:41.016993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.018622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.026940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 360/716] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:41.027872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.037406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544d4a8-1aa0-45b3-804a-da7ab338ecea None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.077787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:41.088169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.089379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.093728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.093728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.107812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.111270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6814d63e-8dbc-4ddb-bc7f-749e2ecbcf77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:41.141629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:41.142580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:41.143483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:41.143483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:41.143483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:41.143483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:41.147484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 357/717] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:41.149322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449ec1-a66f-4ad8-837f-13f3a211c0fa None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.149848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449ec1-a66f-4ad8-837f-13f3a211c0fa None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.150266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449ec1-a66f-4ad8-837f-13f3a211c0fa None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.152291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 358/718] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:41.153426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62449ec1-a66f-4ad8-837f-13f3a211c0fa None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.188056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc5ec-cf23-43c1-bea0-28e2d5ce8634 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.245184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6820043-584e-47c5-bc4e-f763153afb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.245932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820043-584e-47c5-bc4e-f763153afb37 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.246219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820043-584e-47c5-bc4e-f763153afb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.246499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6820043-584e-47c5-bc4e-f763153afb37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.256105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:41.263448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6820043-584e-47c5-bc4e-f763153afb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-e6820043-584e-47c5-bc4e-f763153afb37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:41.272694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:41.274044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 361/719] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:41.276805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.278260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.278260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.299229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:41.301990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:41.309629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62449ec1-a66f-4ad8-837f-13f3a211c0fa None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.352156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.357389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.357389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.357389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.357389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:41.357389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.357389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.369504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 359/720] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:41.370368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.433055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 362/721] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:41.435081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.525778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:41.609838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86c578-48f4-42a7-a75e-76edd2509c12 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.612391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d690-7b0b-4be6-a194-e21c09e37c7c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.645109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:41.653884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.654698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.654977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.655248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.663850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.665709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.683132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:41.683399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:41.685031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 363/722] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:41.686794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.690607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.691312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.691810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.693916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:41.694716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.699104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.712420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 360/723] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:41.713589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.822087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74849-cd71-4a87-b137-148717ddeaa8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.872320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.872634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c71b8-8b98-49e9-bbea-0dd19e95ffeb None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.874077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.874519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.874986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.890955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:41.892519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:41.902590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:41.906716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:41.908459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:41.910095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 364/724] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:41.910167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.948870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.949955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.950478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.950963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.952032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:41.952623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.954538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.970046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 361/725] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:41.971333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.983317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.059908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6d1aa-ad77-41f6-95fb-2ad4a4b95c95 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.101081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.102236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.102730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.103206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.112549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c246289-628f-4ff6-9940-f2b04d4a7009 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.116810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.118607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:42.138373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:42.139348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:42.140255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:42.140255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:42.140255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:42.140255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:42.142586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 365/726] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:42.143683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.150047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.150734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.151020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.151301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.152057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:42.152363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.153771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.163356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 362/727] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:42.164205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.199366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.265058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.271269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b3c65-25d9-4297-af55-91d609f93180 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.281505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00875fdf-586c-4478-8902-33e4331c9fe6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.316416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.318388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.319002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.319523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.320661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=0ceb759c7b4b4292bb9763574d79254b) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:42.321210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.321317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.321888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.322172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.322445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.323168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:42.323472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.324057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.324861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.331318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 366/728] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/role_assignments?user.id=0ceb759c7b4b4292bb9763574d79254b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:42.333611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.338745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 363/729] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:42.351610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.464335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fbbd0-6b65-4328-9b68-77951b69e2b8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.508164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.509431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.509870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.510298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.524041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.526121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-10e49f1c-ef59-43b1-950c-a1dc7e7a94ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:42.547966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:42.548980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:42.550023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:42.550023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:42.550023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:42.550023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:42.553127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 367/730] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:42.554959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bec482-b66d-4009-bb23-713d63363f4e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.555605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bec482-b66d-4009-bb23-713d63363f4e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.556103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bec482-b66d-4009-bb23-713d63363f4e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.558103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 368/731] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:42.560465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bec482-b66d-4009-bb23-713d63363f4e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.596769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.605142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.610963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325a1f0-9ec2-455d-bf84-7149d9e661f9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.653607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.654470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.654754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.655031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.664309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.666371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2f845cd4-6151-45a4-989a-29434b909d0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:42.682731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:42.683769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:42.684779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 364/732] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:42.685943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.686052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.686305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.705870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:42.707162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:42.712667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bec482-b66d-4009-bb23-713d63363f4e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.759268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:42.760476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:42.761174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:42.761685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:42.762628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:42.763104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:42.765056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:42.781746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 369/733] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:42.782979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.833509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 365/734] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:42.834851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:42.965239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.967498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:42.990626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c1eb5-59f4-4ffa-b097-f15da938a8ff None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:42.991195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bba7c0-e5bf-467b-a0ed-990fa080926f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:43.029758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.030546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.030832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.031113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.031956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:43.032271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:43.032593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.033753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:43.033951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.034427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.034942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.045060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:43.046861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:43.053672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 370/735] 10.222.0.58 () {56 vars in 1149 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:43.054493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:43.064474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:43.065471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:43.066389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:43.069977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 366/736] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:43.071314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:43.169790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fa50f-66aa-4f5a-b7ff-4965c7fd0c06 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:43.204602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47260cd-1f84-4169-a1b2-dffca81752a2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:43.218130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.218891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.219233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.219627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.220461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:43.222038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:43.225176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:43.255777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.256684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.256950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.257042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.267500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:43.326816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 371/737] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:29:43 2022] POST /identity/v3/services => generated 251 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:43.328417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:43.345969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:43.347961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:43.516451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565998-2751-4a89-8578-bb580ad7836f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:43.559588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.560700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.561183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.561651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.562701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:43.563214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:43.565132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:43.587545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 372/738] 10.222.0.58 () {56 vars in 1135 bytes} [Fri May 27 03:29:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:43.596571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:43.599270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 367/739] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:43.605573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:43.698869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:43.707230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:43.778044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc69a28-3f33-409b-a1e5-6ecdd18e450f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:43.799187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d47b9-2368-4632-8c7c-12bc8626ebbf None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:43.838608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.841296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.842030 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.842807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.848907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:43.849908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:43.850467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:43.850839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:43.857972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:43.859736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:43.888276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:43.889289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:43.890182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:43.890182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:43.890182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:43.890182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:43.893790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 368/740] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:43.900035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.008567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.010340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.015881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:44.029504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a1aa3-9a11-4bd3-8307-0a91168ca2b1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:44.097292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:44.097990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.098332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.098613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.099431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:44.099747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.101266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.121387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:44.134255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 369/741] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:44.137726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.220445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 373/742] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:43 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:29:44.226849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.351026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedd052-195b-4c71-8bc8-799649cc02b3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:44.361503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8c161-8cd3-4f29-93b4-54dd848b81bb None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:44.406004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:44.406821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.407591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.407874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.416603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.418072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.420300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:44.421849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.422460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.423089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a7330890-9a8d-4451-83e1-a59b6f7742c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:44.434236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:44.436001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:44.437911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 370/743] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:44.438904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490e542-2b47-4a4f-a175-68cdc4185439 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.438904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490e542-2b47-4a4f-a175-68cdc4185439 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.438904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490e542-2b47-4a4f-a175-68cdc4185439 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.439288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.440555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 371/744] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:44.442588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490e542-2b47-4a4f-a175-68cdc4185439 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.444281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.460316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:44.466580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-bc6841e0-7d45-4e3a-ac07-161b0a63be4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:44.477547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:44.478502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:44.479474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:44.479474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:44.479474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:44.479474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:44.487636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 374/745] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:44.490854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.491395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.491918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.518066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:44.519544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:44.580873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8490e542-2b47-4a4f-a175-68cdc4185439 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:44.621296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:44.622300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.622824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.623304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.624305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:44.624872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.626804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.630022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 375/746] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:29:44.631190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.642096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 372/747] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:44.651922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.761659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:44.770929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefcbb-4ab4-41fc-875c-97b2567e5f2d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:44.779062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39ea34-9284-4542-b3a9-0c91eb9de055 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:44.818002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9290213-c938-450b-a798-1673ba501747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:44.819072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9290213-c938-450b-a798-1673ba501747 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.819572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9290213-c938-450b-a798-1673ba501747 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.820086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9290213-c938-450b-a798-1673ba501747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.821171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9290213-c938-450b-a798-1673ba501747 None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=a766bce0474243d2836799b24da5b41a) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:44.821692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9290213-c938-450b-a798-1673ba501747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.823410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9290213-c938-450b-a798-1673ba501747 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.829241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:44.830533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 373/748] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:44 2022] GET /identity/v3/role_assignments?user.id=a766bce0474243d2836799b24da5b41a&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:44.831754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9290213-c938-450b-a798-1673ba501747 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.841405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:44.842725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:44.843207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:44.844229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:44.845318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:44.845913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:44.849081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:44.875917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 376/749] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:44.885800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:44.985848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9290213-c938-450b-a798-1673ba501747 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:45.036593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:45.038210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.038715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.039202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.040182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:45.041904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:45.048599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d1d6-b4e0-455d-ab41-00051515fce5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:45.048913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:45.113075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:45.115008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.115637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.116241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.142783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:45.142783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:45.143122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 374/750] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:29:44 2022] POST /identity/v3/services => generated 240 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-cdbcfafe-beea-4aee-9cf4-667fdb552684 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:45.156562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:45.157540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:45.158475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:45.159416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 377/751] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:45.166608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:45.263109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca32eb-30f1-4843-b53e-78ae3a71df2a None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.263647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca32eb-30f1-4843-b53e-78ae3a71df2a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.265100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca32eb-30f1-4843-b53e-78ae3a71df2a None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.268660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 378/752] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:45.276353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60759888-4bca-451f-b884-05016223411c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.277095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60759888-4bca-451f-b884-05016223411c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.278207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60759888-4bca-451f-b884-05016223411c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.336665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-60759888-4bca-451f-b884-05016223411c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:45.337868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-60759888-4bca-451f-b884-05016223411c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:45.347467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8010f2f-9ccc-40bb-9bcf-1edc675dfcea None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:45.399575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:45.399779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.400035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.400312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.401667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:45.401959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:45.403417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:45.419122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 375/753] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:45.430230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:45.431633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:45.471110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 379/754] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:45.494859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60759888-4bca-451f-b884-05016223411c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:45.516913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:45.605842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c73d77-3fe0-415e-962c-d26bdf907936 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:45.689791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:45.690460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.690736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.691008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.691886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=1c8eaac98b1b4f5db6c0b60c3f01f711) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:45.692205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:45.693694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:45.706829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 376/755] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:45 2022] GET /identity/v3/role_assignments?user.id=1c8eaac98b1b4f5db6c0b60c3f01f711&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:45.709578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60759888-4bca-451f-b884-05016223411c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:45.771214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:45.772048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:45.772333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:45.772641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:45.793260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:45.801999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:45.803064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:45.805253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 380/756] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:45.810218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:45.994645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb05e2-a3a7-4b1a-8782-6735a0627300 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:46.016192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:46.045995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:46.046735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:46.046964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:46.047238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:46.048032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:46.048281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:46.049695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:46.065430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 377/757] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:46.072965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:46.142595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:46.236526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5bfd0-6275-4d9d-9adb-6157f477d1ff None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:46.284269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:46.285034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:46.285321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:46.286198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:46.302639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:46.304120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:46.328055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:46.329044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:46.330382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:46.330382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:46.330382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:46.330382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:46.330382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 381/758] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:46.341605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:46.531975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f08613-a0ae-410a-adb2-c06c12a27a61 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:46.581254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:46.581842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:46.582125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:46.582953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:46.583694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:46.584091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:46.585538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:46.602959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 378/759] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:46.608438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:46.735391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:46.750802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49e57a-c94b-474e-8c65-845b5d7a151b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:46.802187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:46.802990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:46.803317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:46.803579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:46.811783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:46.813288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-223f83ce-7004-45a4-a551-37a7d51c1429 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:46.831877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:46.834468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:46.837312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 382/760] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:29:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:46.839848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:46.852685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03947c-8e4a-4cf0-8a54-3e9558faf48e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:46.853099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03947c-8e4a-4cf0-8a54-3e9558faf48e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:46.853385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03947c-8e4a-4cf0-8a54-3e9558faf48e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:46.854873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 383/761] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:29:46.865373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:46.866432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:46.880900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:46.914458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:46.920655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:46.921873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:47.043240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8aa1c-c366-4112-853e-797ddb36eea7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:47.109798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:47.110823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.110823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.111082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.116555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:47.116555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:47.116555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:47.136464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 384/762] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:47.159041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 379/763] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:29:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:47.164328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:47.191524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:47.356827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:47.372372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefa815-f668-4cdf-8041-66b228bdc2e3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:47.422880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:47.426783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.427333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.427827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.432292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:47.436902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:47.437196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:47.468417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bfbea-f19b-4cf8-9b91-7a5ae2748e93 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:47.484521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:47.485518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:47.486362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:47.489138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 380/764] 10.222.0.58 () {56 vars in 1129 bytes} [Fri May 27 03:29:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:47.492572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:47.530101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:47.530943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.531225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.531496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.570079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:47.572792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:47.625596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:47.626943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:47.627862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:47.627862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:47.627862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:47.627862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:47.627862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 385/765] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:47.637812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:47.740028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b232-a17b-4f08-88bf-d68416d0cd7c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:47.853994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de07e9-100b-4059-aa7b-f3b83e44a063 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:47.869240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:47.870291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.870810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.871286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.872209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:47.872746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:47.874359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850c0be-a198-4201-b38f-cfa7f96693ce None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:47.887510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 381/766] 10.222.0.58 () {56 vars in 1124 bytes} [Fri May 27 03:29:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:47.891916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192500a-3314-45fd-a6ec-2fc2315eb8b2 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.891916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192500a-3314-45fd-a6ec-2fc2315eb8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.891916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192500a-3314-45fd-a6ec-2fc2315eb8b2 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.896206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 382/767] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:47.897889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192500a-3314-45fd-a6ec-2fc2315eb8b2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:47.928426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:47.929166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.929449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.929729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.930466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:47.930778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:47.932417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ba582-6324-45c8-b881-f58f338966ba None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:47.934286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:47.950001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 386/768] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:47.950001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:47.954941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:47.954941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:47.958421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:47.971382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:47.972729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:48.068273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192500a-3314-45fd-a6ec-2fc2315eb8b2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.092992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 387/769] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:48.094815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.105342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.106021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.106315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.106580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.107297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:48.107607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.109052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed75c3-7d08-4131-b718-99af79cf058e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.114484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 383/770] 10.222.0.58 () {56 vars in 1124 bytes} [Fri May 27 03:29:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:48.115861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617abc74-4a57-466d-b928-ea16340a1bf4 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.116152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617abc74-4a57-466d-b928-ea16340a1bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.116453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617abc74-4a57-466d-b928-ea16340a1bf4 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.117757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 384/771] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:48.118476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617abc74-4a57-466d-b928-ea16340a1bf4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.227862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:48.246902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b6da3-a537-439b-bb93-dafc3c2b2d17 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.255560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617abc74-4a57-466d-b928-ea16340a1bf4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.291125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:48.299464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.301023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.301741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.301856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.302655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.303619 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.304440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.304440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.312530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.314066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.320460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-68de2d76-640e-4af6-9b26-7805ad5a6e6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:48.330880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:48.332017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:48.334133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 388/772] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:48.335451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.335451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.335451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:48.345371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:48.349042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 385/773] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:48.351892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.363291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:48.365185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:48.446501 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 389/774] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:48.457455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.530572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774f275-575a-43ff-a5ee-2583a96f0757 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.571526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b7ae4-90ed-4f62-9d0e-af9ebf18bd88 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.574122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.574797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.575083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.575364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.576190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] RBAC: Adding query filter params (interface=public, service_id=e61cf171951e4a55bf9b0faa9eeca818, region_id=RegionOne) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:48.576522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.577959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.587294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 386/775] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/endpoints?service_id=e61cf171951e4a55bf9b0faa9eeca818&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:48.588275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.591703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:48.603408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:48.617036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.619404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.619404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.619404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.619869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:48.620006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.621523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.634914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 390/776] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:48.635817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.715363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044438d-3251-4abd-984f-f0e6012b17b6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.758445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.759131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.759414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.759687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.760442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:48.760779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.762155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9645e6-6486-414d-9533-4f1d7eac5d5b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.771288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 387/777] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:48.772624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5054b9-ac81-413c-9878-1d13cc447a2c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.772915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5054b9-ac81-413c-9878-1d13cc447a2c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.773203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5054b9-ac81-413c-9878-1d13cc447a2c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.774428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 388/778] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:48.777184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501fba4-920e-4a69-ad05-9bdc570eafcd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.777338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5054b9-ac81-413c-9878-1d13cc447a2c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.819887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.820674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.820953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.821232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] PATH_INFO: `/v3/services/network` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.821885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.823262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:48.839502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:48.841468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:48.842767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 391/779] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:48.843720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.872855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:48.896609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:48.909264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5054b9-ac81-413c-9878-1d13cc447a2c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:48.953513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:48.954705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:48.955190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:48.955674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:48.956724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:48.959624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:48.968004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 389/780] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:48.969720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:48.989439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8f3a1-bd2a-4d97-97b1-bd3d6b109b33 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.031188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d7e93-fad3-433d-8cee-c7950b90adf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.032114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7e93-fad3-433d-8cee-c7950b90adf5 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.032412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7e93-fad3-433d-8cee-c7950b90adf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.032721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d7e93-fad3-433d-8cee-c7950b90adf5 None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/0ceb759c7b4b4292bb9763574d79254b/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.034897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d7e93-fad3-433d-8cee-c7950b90adf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=0ceb759c7b4b4292bb9763574d79254b, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.037060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d7e93-fad3-433d-8cee-c7950b90adf5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.107403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0ff22-a7ec-4488-85ad-597eea530f4c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.112324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 392/781] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:48 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/0ceb759c7b4b4292bb9763574d79254b/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:49.114557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.115087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.115587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.141680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:49.143367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:49.182941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.184105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.185395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.185395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.191610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:49.205346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.206106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:49.227289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:49.228277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:49.229209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:49.229209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:49.229209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:49.229209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:49.236552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 390/782] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:49.236552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.285421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 393/783] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:49.286860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.374583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:49.419528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4f368-6a1d-4f8b-82f4-7df7ffbe2fef None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.480180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.482194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.483221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.484459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.486186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:49.487785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.491418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.501906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21738d9-978b-4639-8af2-6d374ad1e0c3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.502428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 391/784] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:49.504798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.559913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.560966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.561393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.561809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.562686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:49.563237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.565033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.578890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 394/785] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:49.579971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.601246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:49.656077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da2b54-ef93-4027-beb0-bc8a0d5d64d9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.690037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:49.703745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc63d5e-03d2-4d40-ba8e-37cc216d9b58 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.715231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.716314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.716771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.717122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.733934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:49.743755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.744739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.745225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.745685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.746568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:49.747048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.748644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:49.749192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:49.750063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:49.751145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:49.757098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 392/786] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:49.758089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.758941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 395/787] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:49.765036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:49.909413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fccc26-4c7c-4cff-93ba-1c2a81e9b8e6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.917294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b7ffd-7e32-4917-a9e4-78f1f7b157a9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:49.932716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:49.955587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.956428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.956765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.957958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.972532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.972532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:49.987253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:49.989050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:49.989381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:49.989778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:49.991102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] RBAC: Adding query filter params (interface=public, service_id=ce9b4775423d4963984c2c9caf805a6f, region_id=RegionOne) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:49.991721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:49.992066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:49.992973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:49.994502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 393/788] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:49.995884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.000795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.022615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 396/789] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/endpoints?service_id=ce9b4775423d4963984c2c9caf805a6f&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:50.024568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.065208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:50.168440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b67a43-e0e6-424e-b8a2-4b99184f205c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.172606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0de82-c3bd-45f9-abc5-89613e97f4ef None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.221629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.223195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.223640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.223812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.224304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.224910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.225390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.225644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:50.225764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.226198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.226945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:50.227266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.228093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.229859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.239034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 394/790] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:50.240147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.278322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 397/791] 10.222.0.58 () {56 vars in 1149 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:50.278322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.288304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:50.384903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:50.399715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b300910-749a-414d-97b2-2de366d7d4eb None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.403466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e61ef-795d-499a-9601-e8f4a148e076 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.448535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.449459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.449664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.449808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.461677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.463141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.469647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.471289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.472063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.472913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.474583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.477637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:50.487018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:50.490640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:50.492822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:50.492822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:50.492822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:50.492822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:50.493281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 398/792] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:50.494072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.494410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 395/793] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/services => generated 772 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:50.496367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.623943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:50.630130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc854703-146c-45ce-9dbd-006bec6899da None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.664049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7575f-055f-433b-8535-aa728f5db2db None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.670239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.671369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.671873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.672347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.673358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:50.673873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.675506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.689068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 396/794] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:50.695558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.740762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:50.758018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.758954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.759350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.759524 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.779133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.780639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:50.811427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:50.812377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:50.813321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:50.813321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:50.813321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:50.813321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:50.821649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 399/795] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:50.828599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.829766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad768-647c-4cdc-b5dc-808b547846d0 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:50.878829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:50.881384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:50.881967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:50.883082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:50.893126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:50.895746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:50.918319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:50.919357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:50.920299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:50.923088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 397/796] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:50.929300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:50.953512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.048899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577e06f-4db1-482a-9a1b-36e84f0c30f6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.070602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940ce51-d052-49c6-b0f2-8d5c15bf85fd None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.099384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.100470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.101016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.101484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.102513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:51.103061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.104889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.124807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.126080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.126562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.127310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 400/797] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:51.127747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.128930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:51.129668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.131323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.135971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.171219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 398/798] 10.222.0.58 () {56 vars in 1135 bytes} [Fri May 27 03:29:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:51.175512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.214887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.312532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.350602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb707-313a-4be9-9ac1-241298ce2580 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.354869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7d00-da78-4203-b2a6-ac700d40c29f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.418511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.419334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.419614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.419887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.430495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.431891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.432794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.433078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.433353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.435757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.460631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.462099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:51.466012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:51.466912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:51.467785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:51.468214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 401/799] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:51.475106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:51.494889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:51.495842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:51.496800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:51.496800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:51.496800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:51.496800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:51.496800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 399/800] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:51.504947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.684880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.694734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55070e-b756-40f5-8fa5-c4535c7f6bb0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.701140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7356805-6d3b-4931-855a-e15eff147de3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.740300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.749845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.750894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.751201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.751613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.752530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:51.753046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.755097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41ea7f-b5c3-40ac-b039-93737e4b1cb4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.755375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.756845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.757339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.757809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.758809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:51.759331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.761036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aacb88-2335-43cb-9660-39f116ca7900 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.772716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 400/801] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:51.772716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83242c6-de40-405c-ac70-47ab65ef8915 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.772716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83242c6-de40-405c-ac70-47ab65ef8915 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.772716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83242c6-de40-405c-ac70-47ab65ef8915 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.776158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 401/802] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:51.777041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83242c6-de40-405c-ac70-47ab65ef8915 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.780251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 402/803] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:51.781973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.782486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.782976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.804104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:51.805281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:51.901344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 403/804] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:51.902187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:51.922974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83242c6-de40-405c-ac70-47ab65ef8915 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:51.972880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:51.973600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:51.973880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:51.974186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:51.982782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:51.984250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2233ae64-140f-4709-8b5c-673567472b70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:51.999412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:52.001018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 402/805] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:52.002215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548f4fb-83a9-422b-bf03-7d272dcb7e5f None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.002771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548f4fb-83a9-422b-bf03-7d272dcb7e5f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.002771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548f4fb-83a9-422b-bf03-7d272dcb7e5f None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.004442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 403/806] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:52.008552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548f4fb-83a9-422b-bf03-7d272dcb7e5f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.031699 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.046968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07f883-780c-4b84-9e88-9e94db812332 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.069010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.100601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26d7ec-984a-4f6a-9d47-ea37242de5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.101492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26d7ec-984a-4f6a-9d47-ea37242de5ae None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.101768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26d7ec-984a-4f6a-9d47-ea37242de5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.102042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26d7ec-984a-4f6a-9d47-ea37242de5ae None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/a766bce0474243d2836799b24da5b41a/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.119064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26d7ec-984a-4f6a-9d47-ea37242de5ae None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=a766bce0474243d2836799b24da5b41a, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.120549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26d7ec-984a-4f6a-9d47-ea37242de5ae None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.133793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548f4fb-83a9-422b-bf03-7d272dcb7e5f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.179030 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.179846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.180139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.180420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.181119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.182539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:52.195230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:52.197225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:52.199717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 404/807] 10.222.0.58 () {56 vars in 1129 bytes} [Fri May 27 03:29:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:52.202449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.216693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 404/808] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:51 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/a766bce0474243d2836799b24da5b41a/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:52.219244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.219834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.220536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.252540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:52.254920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:52.383328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 405/809] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:52.384847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.392261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58db82-f679-4c5c-b368-bfc87c0ef886 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.432042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.458014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.458956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.459431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.459962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.461123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:52.461442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.463404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.483429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 405/810] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:52.483429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.493493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.537508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a15d1-fee4-42c6-94c5-161d49799eab None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.598471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.599597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.600358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.600995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.602025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:52.602765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.604846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.616629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 406/811] 10.222.0.58 () {56 vars in 1124 bytes} [Fri May 27 03:29:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:52.618222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.622087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47739b8a-c564-43b6-b8dc-ae1fdd248737 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.667307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.668123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.668411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.668739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.688836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:52.700170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:52.701204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:52.702044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 406/812] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:52.702729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.747921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e220f9d-5a03-454b-ba14-57f70f105003 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.753249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.783832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:52.797833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.799098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.799739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.800224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/1c8eaac98b1b4f5db6c0b60c3f01f711/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.803604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=1c8eaac98b1b4f5db6c0b60c3f01f711, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.806251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.839943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60356a5f-40aa-4e38-a8e0-49278a828a5e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:52.893827 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 407/813] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:29:52 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/1c8eaac98b1b4f5db6c0b60c3f01f711/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:52.898191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:52.898431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:52.898798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:52.899052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:52.899362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:52.900440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:52.900796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:52.902267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:52.916536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 407/814] 10.222.0.58 () {56 vars in 1124 bytes} [Fri May 27 03:29:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:52.930976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.078369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca778ab-9871-4943-a102-8c53d775390f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.118293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.119399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.119879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.120376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.121394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:53.122058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.123732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.134748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 408/815] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:53.140886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.147605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:53.154393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667d07c-d2dd-4463-ae0b-f0eab0a9000c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.190736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.192140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.192716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.193222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.194689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.197712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.216998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:53.284077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995a9d1-b654-47c7-a7b8-adcc06cf7778 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.296206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 408/816] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:29:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:53.382904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.384241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.384783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.385305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.413454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.416401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9950cd61-4555-4d70-89e9-2d03e05d9e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:53.444756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:53.445715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:53.446584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:53.446584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:53.446584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:53.446584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:53.448391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 409/817] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:53.458434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.534378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618810ab-b111-4510-827a-fa24ad92334f None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.534703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618810ab-b111-4510-827a-fa24ad92334f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.534981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618810ab-b111-4510-827a-fa24ad92334f None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.537056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 410/818] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:53.548600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.548907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.549161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.585704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:53.586734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:53.626012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007ca1-2ca0-452c-bb42-20c16e2417f6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.675083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.675820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.676189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.676634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.677387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:53.677696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.679112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.700490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 409/819] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:53.710375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.730569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:53.736591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 411/820] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:53.780715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:53.826489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:53.865546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3af5d-b245-41e9-9698-231c9e224775 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.936811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:53.937527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:53.937814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:53.938084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:53.948993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:53.950440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:53.958005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a56ed-2682-4983-962b-2652a6ceb26b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:53.972279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:53.973849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:29:53.974934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:53.985272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 410/821] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:53.990287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.026646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.027457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.027738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.028017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.028720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.030101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:54.045502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:54.046572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:29:54.047412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:54.048683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 412/822] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:54.053566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.140229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.200533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dee433-70e7-4b75-a830-341b0a5e7ef8 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.248131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.259965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c58afd-ccd7-4eca-b09f-ab12e2afc7e5 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.274641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.275497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.275770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.276045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.279016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:54.279016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.279016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.318712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 411/823] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:29:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:54.320975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.335391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.336315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.336427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.336659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.337397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] RBAC: Adding query filter params (name=network) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:54.337928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.339322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.359027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 413/824] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:54.366497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.521820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4edaf-46ee-4d0b-81bd-3b9ceaccc597 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.564693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4502e56-7da8-463d-9f40-12a4cfb6b540 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.583054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.593348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.594025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.597981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.597981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.636026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.636736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.637027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.637308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.638027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] RBAC: Adding query filter params (type=network) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:54.638336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.639741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.651080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 414/825] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:54.665447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.697728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.714889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.717188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.786411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3942d-1d11-4d47-9ea4-64a06c31e302 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:54.823623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:54.824296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:54.824612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:54.824895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:54.825638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:54.827034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:54.841076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 412/826] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:29:54.846575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:54.903732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:54.931307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 415/827] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:29:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:55.002639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25da82-86eb-419e-a1f6-df996939c70b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.051175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.051907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.052184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.052482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.065534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.067948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ffe061fc-26d8-41c0-b93d-e61f305f49d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:55.088305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:55.089276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:55.090147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:55.090147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:55.090147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:55.090147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:55.090147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 413/828] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:55.095486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.148109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e09f-a101-4bbc-a55f-228c799c67aa None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.148797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e09f-a101-4bbc-a55f-228c799c67aa None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.149225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e09f-a101-4bbc-a55f-228c799c67aa None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.151110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 414/829] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:29:55.157108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.157630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.158056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.177830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:55.179308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:55.266025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.269669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 415/830] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:29:55.288292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f8171-d7bc-42d6-84b6-368fdd47ddbe None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.294392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.308959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.332632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.333447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.333730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.335087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.335087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:55.335267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.336940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.351390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 416/831] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:55.359034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.447128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb162e33-1a90-4c72-9671-7d5637f00cb5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.498959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.500021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.500329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.500634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.516774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebca2f-3e42-44ec-bbaa-315c80053a76 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.517823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:55.527906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:55.529347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:55.530187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 416/832] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:55.536981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.561824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.563149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.563651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.564241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.574135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.576214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.579744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:55.593498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:55.599357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:55.600282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 417/833] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:55.602270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.628228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.673429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f31338-33f6-4ab2-b4a7-928ac853352c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.712195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.713323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.713614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.713895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.714670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:55.714979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.715462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18183069-534d-40a6-943b-a06426a8fafd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.716371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7cfda-5987-4fd2-b597-2fb09063f2a3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.731356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 417/834] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:55.731356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e3c3b-a368-4bc4-8960-140c90d773a9 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.731356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e3c3b-a368-4bc4-8960-140c90d773a9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.731356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e3c3b-a368-4bc4-8960-140c90d773a9 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.731356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 418/835] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:55.735215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e3c3b-a368-4bc4-8960-140c90d773a9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.761738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.762761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.763321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.764012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.765146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:55.765710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.767622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b5c6-4963-4c34-9916-b734b7cda286 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.781610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 418/836] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:55.783233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.783727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.784206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.802443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:55.803664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:55.853513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e3c3b-a368-4bc4-8960-140c90d773a9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:55.868823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.892565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 419/837] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:55.892565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:55.893307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:55.901229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:55.902054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:55.902708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:55.903042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:55.918706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:55.920184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:55.938176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:55.939268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:55.940239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:55.940239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:55.940239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:55.940239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:55.940239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 419/838] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:55.941023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.011473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50debbf-bd32-4d96-8c6f-7e83ed1d8aad None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.058681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.059694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.060153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.060648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.061492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=0ceb759c7b4b4292bb9763574d79254b) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:56.061918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.063284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.080536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 420/839] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/role_assignments?user.id=0ceb759c7b4b4292bb9763574d79254b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:56.080536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.096469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e3e73-8c34-4dc8-89ad-3414ffa7b141 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.138530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.139267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.139552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.139829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.148368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.154165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:29:56.163752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:56.166212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:56.167050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 420/840] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:56.171992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.218164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db076-50c6-4e6c-8ffc-694617b7a015 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.254666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.264179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.265336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.265591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.265873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.266629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:56.266943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.268491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faba3f2-c16b-4f4d-92aa-14bacc4218c1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.283362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 421/841] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:56.286531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09dc4b-0343-4cb7-88ed-f149e917fd08 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.286928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09dc4b-0343-4cb7-88ed-f149e917fd08 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.287285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09dc4b-0343-4cb7-88ed-f149e917fd08 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.288963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 422/842] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:56.289899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09dc4b-0343-4cb7-88ed-f149e917fd08 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.333487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b99e-dd6d-4ff6-9285-87c81b88c5dd None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.376159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.376885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.377341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.377655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.378953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:56.379273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.380921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48d4ec-6aed-48bb-9407-86ac2bcecd89 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.405749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 421/843] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:56.405749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.405749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.405749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.413637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:56.413825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09dc4b-0343-4cb7-88ed-f149e917fd08 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.414850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:56.431912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.455779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.457500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.457787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.458059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.466173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.467986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:56.487122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:56.488978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:29:56.490368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:56.492621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 423/844] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:56.493650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.498099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 422/845] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:29:56.498579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.543590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.616664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce3cc6-cb40-45b3-9647-0f9eb1f37c23 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.629454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebc5ac-2265-4c0f-b81c-21090c8d55f6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.654312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.654981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.655267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.655553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.656436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:56.656723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.658157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.669440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.670254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.670544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.670822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.677083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 423/846] 10.222.0.58 () {56 vars in 1149 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:56.679390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.682786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.685322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:56.704975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:56.705970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:56.707911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:56.707911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:56.707911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:56.707911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:56.707911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 424/847] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:56.707911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.742396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.794886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:56.805040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23906-13cd-4801-a3ee-65b1860871bf None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.824270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0452c43-890d-4770-ab61-06fce94b76af None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.843024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.843841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.844129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.844406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.845101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.846488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:56.860956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:56.862191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:29:56.863255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:56.870165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 424/848] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:56.870165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:56.875328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:56.875998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:56.876281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:56.876586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:56.961953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:56.963690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:56.985157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5ed83-fa16-4bca-a47d-55e68cee46a4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:56.994616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.023880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.024852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.025138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.025420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.026732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:57.026983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.029310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.042499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 425/849] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:57.043657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.065861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.074616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 425/850] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:29:57.075609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.165179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e250b-5ef9-4baa-8c9a-ef7cc950e2a8 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.197424 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063136-eb3a-4a5e-a32e-ccccefea0eb6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.203072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.203746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.204036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.204309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.205047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:57.205355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.207317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.216988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 426/851] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:57.216988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.234825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.246962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.247782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.248066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.248597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.258098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.259552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:29:57.275821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:57.277136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:29:57.278075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:57.282168 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 426/852] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:57.283060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.313695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.336248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06a66-9084-4789-a4bf-d7dd20c2a68b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.376797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.377808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.377920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.378236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.391152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.393096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.398666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40318f33-e0e3-43ab-b3b2-066eb8858e45 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:57.413502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:57.414647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:57.415536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:57.415536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:57.415536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:57.415536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:57.415536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 427/853] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:57.415828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.444194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.444903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.445188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.445466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.446186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:57.446495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.447951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.453838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 427/854] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:57.454718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.528310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.563866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f882b-2269-4790-8d9f-02f5a0fdbac6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.601623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.608106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.609204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.609652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.613541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.613541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:57.613541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.613541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.616817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464cc1a-1cd4-4600-bcfc-6a030c80b1f6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.638592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 428/855] 10.222.0.58 () {56 vars in 1135 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:57.644645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.665565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.666249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.666533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.666807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.667546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:57.667857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.669310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.684029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 428/856] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:57.689947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.776924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22128-1d1a-4ae3-95e0-e36e226e9ba1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.817380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.818618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.819253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.820034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.826210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf99d1-9b09-4e6d-9b46-13110592d6ef None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:57.841126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.867160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:57.868413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:57.868799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:57.869112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:57.880316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.881980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:57.901633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:57.906249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:57.907192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:57.908398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 429/857] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:57.913968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:57.917550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:57.919566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.054541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344f67d-a5aa-4f23-a1b0-752716eb63bd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.063348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 429/858] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:29:58.065026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.100472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.101193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.101632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.102003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.102808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:58.103191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.105069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.119774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 430/859] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:58.126096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.166005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.179196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.188877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb6867-4d61-419f-b725-e8004ab302c5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.236981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.238826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.239114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.239401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.253546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.255263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.259669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da579f-164f-484f-940f-ac81137d85d3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:29:58.277567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:58.278556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:58.279514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:29:58.279514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:29:58.279514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:29:58.279514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:58.285348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 430/860] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:58.288481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.311082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.311759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.312269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.312576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.313811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=a766bce0474243d2836799b24da5b41a) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:58.314129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.315710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce2250-cd6f-413f-8cd5-574dc9d155f3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.325815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 431/861] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/role_assignments?user.id=a766bce0474243d2836799b24da5b41a&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:58.433170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b22a3-5945-4523-8c6b-5fc022065b58 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.479474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3649a-4844-420d-bbbe-eb43935e3839 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.479928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3649a-4844-420d-bbbe-eb43935e3839 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.480275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3649a-4844-420d-bbbe-eb43935e3839 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.482144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 432/862] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:58.485291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.486040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.486339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.486952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.487708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:58.488840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.489210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.489713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.490044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.490216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.505135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 431/863] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:58.514000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.518506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:58.520300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.540864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.603677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 433/864] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:29:58.631201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.654422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f3929-a20c-4cc5-9426-5b9fe9981f2e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.672334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.718906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.720125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.720439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.720752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.734356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.738802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:29:58.756415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:58.757402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:29:58.758305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 432/865] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:29:58.763491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.791073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29338b83-60b3-4400-9944-bec8b3ed8f61 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.829213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88304cc-e333-438e-b200-914566efde5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.830345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88304cc-e333-438e-b200-914566efde5d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.830839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88304cc-e333-438e-b200-914566efde5d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.831296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88304cc-e333-438e-b200-914566efde5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.832700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88304cc-e333-438e-b200-914566efde5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.834441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88304cc-e333-438e-b200-914566efde5d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.838861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f88304cc-e333-438e-b200-914566efde5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:29:58.849122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:29:58.860478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:29:58.861746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 434/866] 10.222.0.58 () {56 vars in 1132 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:29:58.862701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88304cc-e333-438e-b200-914566efde5d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.888754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbc6c2-9ff6-48fe-8403-31cb8b5598e9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.932824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.934185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.934480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.934765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.935579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:58.935895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.937695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.956013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 433/867] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:58.962795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.969565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.983430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88304cc-e333-438e-b200-914566efde5d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:59.042225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:59.043262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:59.043742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:59.044235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:59.045223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:59.045759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:59.047422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:59.055213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 435/868] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:59.060643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:59.119988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2887b-382e-424a-a766-7ffec775e764 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:59.146542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:59.159445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:59.161459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:59.161459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:59.161735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:59.162692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=1c8eaac98b1b4f5db6c0b60c3f01f711) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:59.163012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:59.164818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793947-4ac9-4b77-8e57-ebaf0fe828da None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:59.177181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 434/869] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/role_assignments?user.id=1c8eaac98b1b4f5db6c0b60c3f01f711&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:59.204032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef6e6e-2ba5-47f9-bf27-a1c90131f950 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:59.247612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:59.248228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:59.248528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:59.248817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:59.249552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:59.250116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:59.251570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f0956-d23b-44fa-b1f0-d687755622ff None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:59.258917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 436/870] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:29:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:29:59.961290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4335c9-df12-4fce-8275-b7b886637328 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:59.961807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4335c9-df12-4fce-8275-b7b886637328 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:59.961883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4335c9-df12-4fce-8275-b7b886637328 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:59.963569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 435/871] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:59.970175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba34849-ce2a-495a-a0c0-c179d040a133 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:59.970740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba34849-ce2a-495a-a0c0-c179d040a133 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:59.970740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba34849-ce2a-495a-a0c0-c179d040a133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:59.987519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bba34849-ce2a-495a-a0c0-c179d040a133 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:59.988852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bba34849-ce2a-495a-a0c0-c179d040a133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:00.004924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:00.095658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 437/872] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:29:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:00.119783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4335c9-df12-4fce-8275-b7b886637328 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:00.229205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4335c9-df12-4fce-8275-b7b886637328 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:00.264322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a43cc-4902-4048-ad3b-4c2e44852f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:00.265034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a43cc-4902-4048-ad3b-4c2e44852f63 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:00.265318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a43cc-4902-4048-ad3b-4c2e44852f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:00.265593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a43cc-4902-4048-ad3b-4c2e44852f63 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:00.266330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a43cc-4902-4048-ad3b-4c2e44852f63 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:00.267698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a43cc-4902-4048-ad3b-4c2e44852f63 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:00.317735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 436/873] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:00 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:00.944589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53011f62-ab43-4e15-aef5-3cadcd9fb679 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:00.945050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53011f62-ab43-4e15-aef5-3cadcd9fb679 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:00.945432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53011f62-ab43-4e15-aef5-3cadcd9fb679 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:00.947258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 438/874] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:00.953208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:00.953639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:00.954022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:00.972712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:00.974271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:01.020926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:01.060647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 437/875] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:01.079177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53011f62-ab43-4e15-aef5-3cadcd9fb679 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:01.189981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53011f62-ab43-4e15-aef5-3cadcd9fb679 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:01.229486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:01.230666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.231181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.231662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.244195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:01.253146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:01.255211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:01.256036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:01.258941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 439/876] 10.222.0.58 () {56 vars in 1120 bytes} [Fri May 27 03:30:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:01.268558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:01.376552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bde5a9-9530-4b62-a986-6d03f8c164e4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:01.417434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:01.418122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.418407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.418683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.419442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:01.419750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:01.421170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae1ca7-2532-4225-88cf-f0cedf7f6659 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:01.435800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 438/877] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:30:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:01.437677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:01.457394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:01.552684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a34278-d135-4ff3-9bb7-48bb6a856c30 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:01.573940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae505f5-2f36-4496-a003-2b218327f429 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.574242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae505f5-2f36-4496-a003-2b218327f429 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.574536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae505f5-2f36-4496-a003-2b218327f429 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.576032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 439/878] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:01.577052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4860c-0228-4b03-992f-add70e542d51 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.577341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4860c-0228-4b03-992f-add70e542d51 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.577620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4860c-0228-4b03-992f-add70e542d51 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.579210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 440/879] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:01.585530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.586193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.586483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.594587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:01.595391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.595675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.595951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.603755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:01.605022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:01.609472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:01.610984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-3c750ec2-8235-442f-a487-499e410a8764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:01.632037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:01.633058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:01.633933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:01.633933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:01.633933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:01.633933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:01.633933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 440/880] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:01.634800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.634800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.635074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.643160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:01.653280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:01.654462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:01.689025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 441/881] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:01.689961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:01.718607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:01.734406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 441/882] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:01.735247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:01.811410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160c28f-d044-44cc-baba-9ca82efeb902 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:01.848085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:01.848882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.849395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.849504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.850142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:01.850449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:01.851819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:01.852213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eec8d-4009-439d-9b9d-fc4c0038bf8e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:01.870108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 442/883] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:01.870108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:01.896807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:01.905511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:01.911077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:01.911077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:01.911077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:01.911077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:01.911077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:01.975687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84b746-2d79-4707-9dd5-22b00462b120 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.009128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.009937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.010216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.010491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.011138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.012534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.021502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 442/884] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:01 2022] POST /identity/v3/services => generated 269 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:02.022771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:02.026142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:02.027230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:02.028103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:02.032798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 443/885] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:02.033832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.107807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:02.163255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:02.181450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb183bc8-ef74-4c98-bd12-4147efc64ff8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.182366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c732b1-8105-4765-b6bb-d51250bbad2b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.232870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.233757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.234045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.234343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.239088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.239984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.240427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.240894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.241771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:02.242217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.243971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.244278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.245470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.250864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 444/886] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:02.257627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:02.265470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:02.266512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:02.267483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 443/887] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:02.272993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.383455 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3b2f8-8749-4fee-a317-edc1ad7b0e32 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.424807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf669b-cdb2-46a6-94e3-9ae173e72d70 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.429612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:02.437351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.438076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.438358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.438632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.439363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:02.440172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.442861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.449944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 445/888] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:02.476074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.476920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.477209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.477487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.478370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:02.478683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.480199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d2155-c730-4b29-a776-4a079b9b3423 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.502754 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 444/889] 10.222.0.58 () {56 vars in 1149 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:02.508762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.531177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:02.640893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2bbf7-3a66-4bc5-9159-117ec18a7a48 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:02.681376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:02.682527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.683009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.683490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.757667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75f8a-9235-417c-b6bf-5a9051b09f29 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.757975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75f8a-9235-417c-b6bf-5a9051b09f29 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.758153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75f8a-9235-417c-b6bf-5a9051b09f29 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.759613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:02.759805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 445/890] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:02.761624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:02.765093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:02.765381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:02.765662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:02.783708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:02.785891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:02.787464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:02.867391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 446/891] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:02.890872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 446/892] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:02.896095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.897765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:02.995642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.006598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21832d-4522-49c2-aed5-f80fbe49ae19 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.007559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d9e89-26c7-4b9b-ab65-327965ae46ed None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.030689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.048968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.050081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.053379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.053488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.053602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.053679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.053749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.053824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.053824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.053956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.063843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.065441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:03.068010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:03.069076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:30:03.069936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:03.072922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 447/893] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:03.082686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:03.086490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:03.087400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:03.088285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:03.088285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:03.088285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:03.088285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:03.092418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 447/894] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:03.093391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.220968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b836520-1cbb-4ab3-b448-5206daa6e64b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.227583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b15d6-6478-44d1-998a-593d8423a352 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.265647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.266424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.266833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.267180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.267672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.268070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:03.268349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.268453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.268658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.269670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.270599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:03.270707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.270909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.272646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.279601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 448/895] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:03.285286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.288645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 448/896] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:03.294102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.320002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.328986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.407736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12df40b-b3fa-4197-bb4f-589668250982 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.426459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72391b72-672b-4bd3-9289-42789ae4360f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.458597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.460033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.460553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.461066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.463373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.464578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.465018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.465572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.466895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:03.467515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.469420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.469797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.472676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.483545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 449/897] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:03.484726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:03.500005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:03.501161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:03.502218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:03.505337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 449/898] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:03.512323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.609919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.626612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:03.637961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c691158-6f78-45e0-b47f-d614020a9f26 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.648161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932726-0d1a-418b-80b2-f6b4e2d70354 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.678560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.679067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.679361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.683065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.683065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] RBAC: Adding query filter params (interface=public, service_id=0905afcfd83d478fb05507c58a1ac013, region_id=RegionOne) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:03.683065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.683065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.689256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.690287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.691643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.692106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.693133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:03.693618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.694764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 450/899] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/endpoints?service_id=0905afcfd83d478fb05507c58a1ac013&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:03.695612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.695934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.713773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 450/900] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:03.719214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:03.819048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f2e5-fa64-4f6d-b9ee-1826b984e189 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.838978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384b703-3374-4ffe-aedb-5057de50c6ef None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:03.858595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036ddc8-326b-482d-aa9e-7a582d4b8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.859268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036ddc8-326b-482d-aa9e-7a582d4b8263 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.859553 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036ddc8-326b-482d-aa9e-7a582d4b8263 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.859829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036ddc8-326b-482d-aa9e-7a582d4b8263 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.860592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036ddc8-326b-482d-aa9e-7a582d4b8263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.863047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036ddc8-326b-482d-aa9e-7a582d4b8263 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.888593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 451/901] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:03.888829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:03.888829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:03.888829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:03.888829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:03.889166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] RBAC: Adding query filter params (role.id=ff0061288bd74b86a760a56c6f67ded2, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=a766bce0474243d2836799b24da5b41a) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:03.889166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:03.889559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e759e-c8aa-41ea-97ec-373969e4e246 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:03.897473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 451/902] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:03 2022] GET /identity/v3/role_assignments?user.id=a766bce0474243d2836799b24da5b41a&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=ff0061288bd74b86a760a56c6f67ded2 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:04.518363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070b6e4-3854-4294-ab2c-511bb2f886b7 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.519062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070b6e4-3854-4294-ab2c-511bb2f886b7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.519062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070b6e4-3854-4294-ab2c-511bb2f886b7 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.520819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 452/903] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:04.526734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.527216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.527293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.545976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:04.547519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:04.556871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:04.632606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 452/904] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:04.651798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070b6e4-3854-4294-ab2c-511bb2f886b7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.669339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:04.811376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070b6e4-3854-4294-ab2c-511bb2f886b7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:04.852494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:04.853741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.854227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.854704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.855598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:04.857220 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f88444ac-6ae2-4423-b8b6-c6a72d13d06b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:04.886072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:04.887035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:30:04.887862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:04.895858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 453/905] 10.222.0.58 () {56 vars in 1132 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:04.898128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:04.933648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287026e-8c3d-4a3d-8cdd-b29211568fe6 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.934008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287026e-8c3d-4a3d-8cdd-b29211568fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.934430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287026e-8c3d-4a3d-8cdd-b29211568fe6 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.936119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 454/906] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:04.942266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:04.942550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:04.942824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:04.969360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:04.970594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.026085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e8c9-5fd7-414b-8c5f-e912f9d902af None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.069072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.071081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.071562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.071925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.072756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:05.073136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.074766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.086335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 453/907] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:05.092297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.103900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:05.108763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 455/908] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:05.144297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.155363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:05.221019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32591-c8b5-451d-9c08-8ab840dbcf90 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.259269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.260153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.260522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.260865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.261684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:05.262106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.263788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.266604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e780130-5ab6-427a-ae94-2f47ce999581 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.275023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 454/909] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:05.276334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.305922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.306965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.307425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.307877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.309016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.310669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.354995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:05.387397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 456/910] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:05 2022] POST /identity/v3/services => generated 241 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:05.389424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea55336-97a9-4b23-ad1b-35069c80a0b6 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.437769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.438494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.438809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.439113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.439925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] RBAC: Adding query filter params (interface=public, service_id=625670bfe6384ce8bf412fa2fd051d58, region_id=RegionOne) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:05.440309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.442380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e191180-07fc-4159-9b52-556e3005262e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.452668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 455/911] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/endpoints?service_id=625670bfe6384ce8bf412fa2fd051d58&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:05.454782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.539061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:05.583858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c337e3-c868-40a7-9491-cb0da76d2254 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.623953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13747a7c-3e14-4a83-bb42-3fbbaf3e94ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.624841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13747a7c-3e14-4a83-bb42-3fbbaf3e94ef None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.625132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13747a7c-3e14-4a83-bb42-3fbbaf3e94ef None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.625412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13747a7c-3e14-4a83-bb42-3fbbaf3e94ef None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.626239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13747a7c-3e14-4a83-bb42-3fbbaf3e94ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.627631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13747a7c-3e14-4a83-bb42-3fbbaf3e94ef None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.636650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 457/912] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:06.315140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea911b5d-71ae-4efb-bd8f-edef6a1219e3 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.316255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea911b5d-71ae-4efb-bd8f-edef6a1219e3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.316255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea911b5d-71ae-4efb-bd8f-edef6a1219e3 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.318100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 456/913] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:06.324034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.324351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.324639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.338931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677bbf3-d10a-4877-9fb2-8a4c66e21a1e None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.339265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677bbf3-d10a-4877-9fb2-8a4c66e21a1e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.339911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677bbf3-d10a-4877-9fb2-8a4c66e21a1e None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.341676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 457/914] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:30:06.345954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:06.346763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.347061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.347348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.348096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:06.369516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:06.379061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:06.452532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 458/915] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:06.468628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 458/916] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:06.469814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.494146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.553286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:06.574107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:06.604033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd0d16-3e82-4b9d-a4e5-d60062e6b638 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.620785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90228fb6-5e85-48f8-b925-7dad18e814de None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.645975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.646791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.647078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.647357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.648025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.649591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:06.664270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:06.665396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:30:06.666228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:06.670689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 459/917] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:06.671555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.671956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.672874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.673352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.673829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.687735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:06.697150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:06.698159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:06.698988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:06.707178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 459/918] 10.222.0.58 () {56 vars in 1120 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:06.708689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.812708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6576de-f80e-41b9-baa9-0614bb239246 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.831765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d33e09-3fbe-404f-9664-1c9ba44e5ac5 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.842068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:06.856730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:06.857746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.858623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.859064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.859487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.860392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:06.860886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.862995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.873725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.874986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.881508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.881508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.881508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:06.881508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.881508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:06.891868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 460/920] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:06.894645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.894777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 460/920] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:06.897745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.009974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49479b4-91de-409c-9c7e-fdf65467e6bc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.017659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab709a6-31ee-4017-ba1b-36c0bd38cd6f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.049126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.050769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.051147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.051606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.059582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.060484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.060946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.061397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.062350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:07.062826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.064332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.064572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.066894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.076635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 461/921] 10.222.0.58 () {56 vars in 1114 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:07.078036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:07.087003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:07.087939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:07.089231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:07.089231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:07.089231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:07.089231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:07.092751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 461/922] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:07.094555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.131942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:07.142760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:07.195095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728429f-b0ba-4b7f-aa5e-9a7699656743 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.204747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38829d8-c302-4282-8c13-7855b03991cb None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.232774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.233615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.234003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.234474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.235572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.236949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.244753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.245853 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.246279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.246804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.247712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:07.248233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.249898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.263672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 462/923] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:07.269255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.327584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 462/924] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:07.389655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:07.397027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f1ee4-6822-496c-9c32-7465f3cc0bd7 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.444877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.445848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.446132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.446408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.454876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.456525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:07.474516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:07.475551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:07.476951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 463/925] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:07.482227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.605200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd223c-bcfe-4989-8602-5662e246e230 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.615701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:07.642658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.643266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.645725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.645725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.645996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:07.646234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.647730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9801e8-c85f-4cc9-a578-edd9723c2bf2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.668831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 463/926] 10.222.0.58 () {56 vars in 1149 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:07.675114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.763455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60333fd6-a273-4194-a89e-83989d7cadb0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.763820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60333fd6-a273-4194-a89e-83989d7cadb0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.764157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60333fd6-a273-4194-a89e-83989d7cadb0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.766195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 464/927] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:30:07.773268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.773681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.773990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.792409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:07.793775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:07.812357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a979271-8975-4514-871a-da33aa16f866 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.876769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 465/928] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:07.884079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.884924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.885212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.885491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.897174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.898146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.899628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:07.919363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:07.920387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:07.921374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:07.921374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:07.921374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:07.921374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:07.927792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 464/929] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:07.929171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.931995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.003915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.010008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee683369-d8cb-468f-84bf-40d1babdc9a8 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.050648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b302f-21e6-4d92-bbc2-f3f9de45890f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.052663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.053805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.054209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.054488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.055192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.056640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-22db6ff3-79f9-4cfe-9344-7ae05fc15088 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:08.070013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:08.071109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:08.071958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 466/930] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:08.072809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecfda-d6ab-4d6c-b102-7189f1435234 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.073968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecfda-d6ab-4d6c-b102-7189f1435234 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.073968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ecfda-d6ab-4d6c-b102-7189f1435234 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.074798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 467/931] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:08.079821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecfda-d6ab-4d6c-b102-7189f1435234 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.107984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.108704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.108995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.109274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.110028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:08.110347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.111793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110a691-21ec-4c27-9b24-e8a4a792c141 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.132297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 465/932] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:08.132297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.132297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.132297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.150202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:08.151346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.215370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ecfda-d6ab-4d6c-b102-7189f1435234 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.223314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.239207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 466/933] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:08.242415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.281019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.282001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.282456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.282922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.283648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:08.283964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.285840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.292482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 468/934] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:08.297899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.328726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.387602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1a0f4-8b5d-4006-b39c-c2c7479a94c4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.422165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ded823-4998-4e28-9684-d7c9575dff87 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.427854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.428844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.429385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.429385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.441259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.442835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:08.458303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:08.459992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 467/935] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:08.461626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.483061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.483889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.484180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.484461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.485156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.486548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:08.500186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:08.501898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:08.503724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 469/936] 10.222.0.58 () {56 vars in 1132 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:08.505326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.532393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.594488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d82963-e448-4373-82ea-a5fede4db72d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.615636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.630805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e5b0b-8ce5-4c7b-83d5-43ba4a8938fd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.637855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.639098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.639392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.639684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.640746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:08.641060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.642478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.657361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 468/937] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:08.657361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.695015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.696228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.696834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.697330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.698796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:08.699341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.702060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.721091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 470/938] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:08.722972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.804594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2ece8-0230-4cdd-b2de-ed57144ef632 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.822998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.843916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.844615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.844909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.845185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.845899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:08.846210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.848264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.853234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 469/939] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:08.853912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.857736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154882d9-71d1-4396-8c5a-b3894645a92c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.907648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.908508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.908877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.909847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.910716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] RBAC: Adding query filter params (interface=public, service_id=6b831d436668470c81d2d98c9cf66bcc, region_id=RegionOne) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:08.911114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.912851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.923751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 471/940] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/endpoints?service_id=6b831d436668470c81d2d98c9cf66bcc&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:08.925235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.941472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.972906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbaae3-2b57-448f-a17a-cb792c5c35bf None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.009598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299686-639d-479f-93c2-34346582d51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.010496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299686-639d-479f-93c2-34346582d51e None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.010781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299686-639d-479f-93c2-34346582d51e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.011062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc299686-639d-479f-93c2-34346582d51e None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/a766bce0474243d2836799b24da5b41a/roles/ff0061288bd74b86a760a56c6f67ded2` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.013380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc299686-639d-479f-93c2-34346582d51e None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=a766bce0474243d2836799b24da5b41a, role_id=ff0061288bd74b86a760a56c6f67ded2)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.014836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc299686-639d-479f-93c2-34346582d51e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.038527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e42944-0f96-4752-8d88-52de9c120e57 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.075138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.075821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.076125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.076400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.077146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:09.077456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.078845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.084854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 472/941] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:09.085641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.135370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 470/942] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:30:08 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/a766bce0474243d2836799b24da5b41a/roles/ff0061288bd74b86a760a56c6f67ded2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:09.136780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299686-639d-479f-93c2-34346582d51e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.178233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:09.220047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39293f-d1f9-44d9-a9e9-350ba2427728 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.228158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:09.263393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fac2a1-360f-492a-9ebd-c26f66f914d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.264526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fac2a1-360f-492a-9ebd-c26f66f914d8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.264526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fac2a1-360f-492a-9ebd-c26f66f914d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.264806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fac2a1-360f-492a-9ebd-c26f66f914d8 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.265522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fac2a1-360f-492a-9ebd-c26f66f914d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.266948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fac2a1-360f-492a-9ebd-c26f66f914d8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.275421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 473/943] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:09.310208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc299686-639d-479f-93c2-34346582d51e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.357240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdfbac-d56e-4d7c-8b14-1b8a1cf688e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.358511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdfbac-d56e-4d7c-8b14-1b8a1cf688e9 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.359126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdfbac-d56e-4d7c-8b14-1b8a1cf688e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.359859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdfbac-d56e-4d7c-8b14-1b8a1cf688e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.361239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdfbac-d56e-4d7c-8b14-1b8a1cf688e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.362978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdfbac-d56e-4d7c-8b14-1b8a1cf688e9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.489990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 471/944] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:09.812866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a0b5c-944e-4e9c-9994-3b8bea16eaed None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.813359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a0b5c-944e-4e9c-9994-3b8bea16eaed None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.813359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a0b5c-944e-4e9c-9994-3b8bea16eaed None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.814943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 474/945] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:09.821427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.821737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.821996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.839896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:09.841502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.926976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 472/946] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:09.951011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a0b5c-944e-4e9c-9994-3b8bea16eaed None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.067265 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a0b5c-944e-4e9c-9994-3b8bea16eaed None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.102610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.103267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.103557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.103834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.104579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.106045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.107440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING keystone.server.flask.common [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:30:10.119097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:10.168344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.sql.core [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:30:10.168344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:30:10.168344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [parameters: {'id': '129fc1b5f99f4e938b4ea3203050c2b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:30:10.168344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80701) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 129fc1b5f99f4e938b4ea3203050c2b8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 129fc1b5f99f4e938b4ea3203050c2b8. May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application conn.query(q) May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:30:10.185164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application result.read() May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 03:30:10.186034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application next(self.gen) May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application yield resource May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application next(self.gen) May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise self.value May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application session.commit() May 27 03:30:10.186991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self.session.flush() May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application compat.raise_( May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:30:10.187887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:30:10.191221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application conn.query(q) May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application result.read() May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:30:10.192042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [parameters: {'id': '129fc1b5f99f4e938b4ea3203050c2b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:10.197770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:10.199633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 129fc1b5f99f4e938b4ea3203050c2b8. May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:10.201798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 475/947] 10.222.0.58 () {60 vars in 1140 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:10.204841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.243452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:10.334725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17811e5-6987-4238-85d2-d5dfea77f80c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.374493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.375695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.376176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.376683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.392264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.394787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-4b007967-d56c-4c8d-953c-188bf9e8b60f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:10.410536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:10.411518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:10.412623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:10.415330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 473/948] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:10.421041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.573271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94ec3e-feb0-45c6-8d6f-e547fe09d4e8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.611963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.613151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.613571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.613916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.614756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:10.615143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.616918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c4971-010a-4450-8967-6a60eb91d62d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.640183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 476/949] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:11.752544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220aab-8fad-4cd3-b1dd-275d76fe3d20 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.752544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220aab-8fad-4cd3-b1dd-275d76fe3d20 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.755600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220aab-8fad-4cd3-b1dd-275d76fe3d20 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.756090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 474/950] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:11.762323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.763310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.763806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.784741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:11.786939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:11.844602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580e6e3-497c-469b-87b3-21fa69bb3932 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.845224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580e6e3-497c-469b-87b3-21fa69bb3932 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.845747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580e6e3-497c-469b-87b3-21fa69bb3932 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.851579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 475/951] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:11.853301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc083f-0e44-493f-91fd-4773c2bec4f2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.853820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc083f-0e44-493f-91fd-4773c2bec4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.854301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc083f-0e44-493f-91fd-4773c2bec4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.861864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:11.872555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 477/952] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:11.883335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6dcc083f-0e44-493f-91fd-4773c2bec4f2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:11.890118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6dcc083f-0e44-493f-91fd-4773c2bec4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:11.900674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:11.988247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 476/953] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:11.989993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285d948-aa36-40e8-860b-82d00681839a None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.990444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285d948-aa36-40e8-860b-82d00681839a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.990903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285d948-aa36-40e8-860b-82d00681839a None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.992407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 477/954] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:12.002417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.002974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.003434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.021354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:12.022697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:12.031127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace6c55-3cb5-4e82-af6b-cc49a11da5c6 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.031707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.073615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.074842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.075126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.075481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.076095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.077596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:12.091227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:12.092401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:12.093313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 478/955] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:12.094939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.109599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 478/956] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:12.110344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.157380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.218883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e21aa6-4814-468d-b551-56f5af4d234b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.220766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32882848-baf8-4b49-a69f-24261dd5f578 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.256986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.257174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.257568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.257736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.257855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.258004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.258107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.258276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.258810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:12.259117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.260486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.265476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 479/957] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:12.266252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.276029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:12.276708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:12.284745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:12.285772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:12.286693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 479/958] 10.222.0.58 () {56 vars in 1120 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:12.287463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.388089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9de2e2-2442-466a-b2fa-d897299a8b34 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.392477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2546c-d738-47ce-bb3f-b2244aa7a833 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.403798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.428129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.428826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.429110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.429382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.430091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:12.430397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.430975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.431776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.432227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.432744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.433231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.434134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.435580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.437332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 480/959] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:12.437965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:12.453072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:12.454145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:12.455021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 480/960] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:12.456414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.541696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.553290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea2852-24ad-4a40-a5ea-24c83761f029 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.577499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a7417-21d2-4309-8409-b8d27ad47e9f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.598610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.599607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.600018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.600466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.601413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:12.601865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.603825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.614464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 481/961] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:12.615384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.627536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.629346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.629842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.630390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.631660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.633676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.664087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.723102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 481/962] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:30:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:12.724282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.747555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8df38-65fc-4a8e-80e1-df308fc7f750 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.802610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.804153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.804153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.804153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.805060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:12.805477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.806780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.812771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 482/963] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:12.821958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.870122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3b6b-ab29-4016-9260-55f0752dd7dc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.905864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.929769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.931312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.932281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.933066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.955688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.958349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:12.980895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:12.981976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:12.982950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:12.982950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:12.982950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:12.982950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:12.982950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 482/964] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:12.986484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.988241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.996001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bec14-f25b-4b3c-8f8a-f8207e7fa24e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.055113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.055709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.055994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.060522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.060522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:13.060522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.060522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.064391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 483/965] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:13.129508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9e3f-584e-498f-8fe8-b31771487ff9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.172984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.173685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.173974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.174258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.175616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:13.175922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.177413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb8fe2-748c-4add-943e-54367b8caa76 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.192698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 483/966] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:13.198357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.231560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602f464-8c60-4531-843f-3be46f241153 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.231887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602f464-8c60-4531-843f-3be46f241153 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.232158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602f464-8c60-4531-843f-3be46f241153 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.235071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 484/967] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:30:13.240196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.240546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.240794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.275768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:13.275768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:13.318413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:13.325842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347488a3-63b7-459b-a3ac-df58e5eba2fd None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.376270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 485/968] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:13.381444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.382568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.383105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.383597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.391914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.393798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.403002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:13.413180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:13.414172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:30:13.415125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:13.418074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 484/969] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:30:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:13.424255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.453756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:13.532025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740db77-78d2-49cb-8481-4636cf8e742a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.568363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9b9d0-84fe-4a09-8161-e5ec5ab95ab7 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.577472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.578538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.579039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.579583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.583867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.589187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.590850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING keystone.server.flask.common [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:30:13.608944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.609628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.609913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.610188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.611065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:13.614247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.614247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.634471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 485/970] 10.222.0.58 () {56 vars in 1149 bytes} [Fri May 27 03:30:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:13.639973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.646862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.sql.core [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:30:13.646862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:30:13.646862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [parameters: {'user_id': '1830b4923eec4c4e8155ee57b4ebb251', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:30:13.646862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80700) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application conn.query(q) May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result.read() May 27 03:30:13.655738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:30:13.656699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application next(self.gen) May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application yield resource May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application next(self.gen) May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise self.value May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application session.commit() May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:30:13.657698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.session.flush() May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application compat.raise_( May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:30:13.658589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:30:13.660612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application conn.query(q) May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application result.read() May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:30:13.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [parameters: {'user_id': '1830b4923eec4c4e8155ee57b4ebb251', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:13.662741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 03:30:13.663737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 03:30:13.664838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 03:30:13.664838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 03:30:13.664838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:30:13.664838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:30:13.664838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:30:13.664838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:13.672208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 486/971] 10.222.0.58 () {60 vars in 1131 bytes} [Fri May 27 03:30:13 2022] POST /identity/v3/users => generated 160 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:13.674120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.717137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:13.738267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:13.760111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e05ff-95ef-4c6f-899c-e710ab4bdcc2 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.799633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.800583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.800970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.801350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.805981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33285d-cd77-416a-a46c-e3d5d2be4fc1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.855256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.856685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.857228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.857648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.870892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.875650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.891554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.893800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:13.897337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:13.898409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:30:13.899562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:13.906701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 487/972] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:30:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:13.908325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.986747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:14.020480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 486/973] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:30:14.021625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.043235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:14.056315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580dd3a-8672-4216-83fa-55b7f0e14a73 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.110384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.111661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.112936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.113476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.115161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:14.115845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.118748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.177230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172f7d4-190f-4cca-a290-b1d06b97ed5c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.216529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 488/974] 10.222.0.58 () {56 vars in 1101 bytes} [Fri May 27 03:30:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:14.228043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.228977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.229267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.229545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.243037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.244567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:14.287197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:14.288544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:14.289490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:14.289490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:14.289490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:14.289490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:14.289490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 487/975] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:14.295104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.434146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a2f0d-7a61-47d8-8bef-c208c85cca25 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.479033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.480136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.482352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.483450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.484543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:14.484871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.491198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.514855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 489/976] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:14.521951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.559346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:14.681217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456578c7-37e2-4e43-81fd-cf29c0bcdb5d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.747603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.748916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.749732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.750239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.763193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.766152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:14.795330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:14.796363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:14.797292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:14.803250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 488/977] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:14.805111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.819980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:14.925955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85313e-8751-4fab-98dc-aa5a65088e1d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.966337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.967024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.967308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.967584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.968465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:14.968869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.970565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4831-2682-4676-94cb-1566d60c7b0a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.985203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 490/978] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:14.992026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:15.096910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:15.106209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367031-21f1-4c0e-ab38-ff23e18b83c4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.146322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.146983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.147260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.147531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.148360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] RBAC: Adding query filter params (role.id=ff0061288bd74b86a760a56c6f67ded2, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=a766bce0474243d2836799b24da5b41a) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:15.148695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.151455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530435-9493-4812-b4b7-82bce0f4ba44 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.162789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 489/979] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/role_assignments?user.id=a766bce0474243d2836799b24da5b41a&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=ff0061288bd74b86a760a56c6f67ded2 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:15.169174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e3f58-a823-434f-a89b-854bb100a580 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.169465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e3f58-a823-434f-a89b-854bb100a580 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.169747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e3f58-a823-434f-a89b-854bb100a580 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.171365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 491/980] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:15.177389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51552223-a51c-4818-ba7d-7b061c7e8cb1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.177695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51552223-a51c-4818-ba7d-7b061c7e8cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.177956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51552223-a51c-4818-ba7d-7b061c7e8cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.196889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-51552223-a51c-4818-ba7d-7b061c7e8cb1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:15.198121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-51552223-a51c-4818-ba7d-7b061c7e8cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:15.282574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 490/981] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:15.308735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e3f58-a823-434f-a89b-854bb100a580 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:15.407224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:15.452346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e3f58-a823-434f-a89b-854bb100a580 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.508536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.508536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.508853 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.509109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.512524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.512524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.527659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9a609-66d4-47c1-8911-ac97d25b2bad None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.527659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9a609-66d4-47c1-8911-ac97d25b2bad None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.527659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9a609-66d4-47c1-8911-ac97d25b2bad None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.527659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 491/982] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:15.528546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:15.529514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:15.530728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 492/983] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:15.533218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.533518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.533801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.541546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:15.566514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:15.568282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:15.676560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 492/984] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:15.695686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:15.717249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd645cf-9ca8-4e69-91ec-d4968e62bae1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.723770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:15.770175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:15.777387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.778130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.778424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.778698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.779540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:15.779855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.781387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.793698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 493/985] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:15.794748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:15.880906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82bc2a-e943-4cee-9718-0cc1baf92f99 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.932437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21942-c171-44a6-a8c0-a25a5607f22b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.968532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.972538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.972538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.972538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.972538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.972538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.980837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.981601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.981955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.982243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.983133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:15.983329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.984743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.992871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 494/986] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:15.998489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:16.048193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 493/987] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:15 2022] POST /identity/v3/services => generated 242 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:16.079967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:16.137545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260e05b-0b94-4064-9700-4d252c61a1ae None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:16.185780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:16.186812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:16.187304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:16.187797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:16.188851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] RBAC: Adding query filter params (interface=admin, service_id=6b831d436668470c81d2d98c9cf66bcc, region_id=RegionOne) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:16.189392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:16.191094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a370a-9286-42a6-838b-b85c0bea268d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:16.197993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 495/988] 10.222.0.58 () {56 vars in 1249 bytes} [Fri May 27 03:30:15 2022] GET /identity/v3/endpoints?service_id=6b831d436668470c81d2d98c9cf66bcc&interface=admin®ion_id=RegionOne => generated 193 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:16.203386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:16.320532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:16.354477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16c1a7-ab09-4949-8cc6-e745de1709d3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:16.396154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0be32-dea1-4e63-9edd-08325e3e4f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:16.397144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0be32-dea1-4e63-9edd-08325e3e4f79 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:16.397429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0be32-dea1-4e63-9edd-08325e3e4f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:16.397703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0be32-dea1-4e63-9edd-08325e3e4f79 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:16.398466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0be32-dea1-4e63-9edd-08325e3e4f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:16.399852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0be32-dea1-4e63-9edd-08325e3e4f79 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:16.414386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 494/989] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:30:16 2022] GET /identity/v3/services => generated 1727 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:17.604305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdc1ac-213c-48fb-8fbc-77fef5367d82 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:17.604886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdc1ac-213c-48fb-8fbc-77fef5367d82 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:17.604886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdc1ac-213c-48fb-8fbc-77fef5367d82 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:17.606286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 496/990] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:17.611441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:17.611882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:17.612478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:17.630253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:17.631422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:17.704304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 495/991] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:17.726781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdc1ac-213c-48fb-8fbc-77fef5367d82 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:17.841394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdc1ac-213c-48fb-8fbc-77fef5367d82 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:17.863164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:17.884536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:17.885681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:17.886002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:17.886324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] PATH_INFO: `/v3/services/image` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:17.886990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:17.888354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:17.900923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:17.902084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:17.903033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:17.908337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 497/992] 10.222.0.58 () {56 vars in 1105 bytes} [Fri May 27 03:30:17 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:17.908674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:17.985907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:18.022968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0f8d-7331-4c53-bb2d-334d07a4a06a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:18.059045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:18.059718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.059995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.060264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.061000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:18.061308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:18.062669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833da34e-cbb0-4790-8c0b-c1998af169a5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:18.072771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 496/993] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:18.074537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:18.192481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d4bae-0c8b-41b9-abfd-bbed09fbff3f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:18.233447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:18.234191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.234451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.234737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.235464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] RBAC: Adding query filter params (type=image) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:18.235776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:18.237804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e80439-1100-46d6-9db6-14a31f53b693 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:18.248824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 498/994] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:18.505300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84040dc1-7758-46b0-8d8c-78cf6a6e7248 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.505489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84040dc1-7758-46b0-8d8c-78cf6a6e7248 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.505757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84040dc1-7758-46b0-8d8c-78cf6a6e7248 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.507229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 497/995] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:18.513492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2cfaf-cd58-4994-a4eb-a6413fc04853 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.513808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2cfaf-cd58-4994-a4eb-a6413fc04853 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.514066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2cfaf-cd58-4994-a4eb-a6413fc04853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.533840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-77e2cfaf-cd58-4994-a4eb-a6413fc04853 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:18.536057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-77e2cfaf-cd58-4994-a4eb-a6413fc04853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:18.588203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:18.612714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 499/996] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:18.636041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84040dc1-7758-46b0-8d8c-78cf6a6e7248 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:18.703903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba65687-4648-4519-9f7f-22571a43b257 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.704346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba65687-4648-4519-9f7f-22571a43b257 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.704699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba65687-4648-4519-9f7f-22571a43b257 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.707851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 500/997] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:30:18.713946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.714291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.714520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.735666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:18.737729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:18.759738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84040dc1-7758-46b0-8d8c-78cf6a6e7248 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:18.797977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:18.804577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:18.805393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.808893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.808893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.808893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:18.808893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:18.818546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 501/998] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:18.821200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:18.827531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:18.828489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 498/999] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:18.829319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:18.843916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:18.970124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2466a7-cd8c-4991-a2f3-48926416f7eb None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:18.982018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65063335-4e76-4b37-8728-e41f99e812c1 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:18.987155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.016710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.017488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.017846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.018135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.019918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:19.020332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.021690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.021953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.022748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.023040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.023322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.023989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.025418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.028374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 502/1000] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:19.034510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:19.056236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:19.057386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:30:19.058241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:19.059000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 499/1001] 10.222.0.58 () {56 vars in 1126 bytes} [Fri May 27 03:30:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:19.064360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.142102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.186918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1316000-ca59-45a1-b509-7ed0e99697c4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.201498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa08e-24e1-44f4-ae56-5a848b8c0d35 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.229677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.230282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.230570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.230849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.231562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:19.231872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.233279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.243689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 503/1002] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:19.244956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.262515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.263572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.263868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.264149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.264896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:19.265208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.266997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.277017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 500/1003] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:19.279210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.298767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.386408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12cd46-021e-4055-ac7f-fc4d80302334 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.445521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.446276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.446681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.447013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.447963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] RBAC: Adding query filter params (interface=public, service_id=7e75afe8edde40288bb0e0e83e5313ab, region_id=RegionOne) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:19.448278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.449753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.450955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f80e4-48ab-4ecd-9323-400c2fbe6a9f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.461246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 504/1004] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:30:19 2022] GET /identity/v3/endpoints?service_id=7e75afe8edde40288bb0e0e83e5313ab&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:19.462474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.486779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.500712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.501496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.501790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.503288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.503398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:19.503720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.505211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.511637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 501/1005] 10.222.0.58 () {56 vars in 1122 bytes} [Fri May 27 03:30:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:19.517281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.596008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.600256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd389bb-b3b9-4945-80d6-16f06df5aaca None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.643084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285734a-11b1-4263-bdd1-ca564a689a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.645028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285734a-11b1-4263-bdd1-ca564a689a7a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.645028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285734a-11b1-4263-bdd1-ca564a689a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.645028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285734a-11b1-4263-bdd1-ca564a689a7a None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.645696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285734a-11b1-4263-bdd1-ca564a689a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.647085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285734a-11b1-4263-bdd1-ca564a689a7a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.647434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6546-2fbb-446d-b44e-1773b5fbddef None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.654590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 505/1006] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:30:19 2022] GET /identity/v3/services => generated 1727 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:19.687369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ec0b8-c229-41c5-8e62-f0946edfcc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.688053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec0b8-c229-41c5-8e62-f0946edfcc8b None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.688335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec0b8-c229-41c5-8e62-f0946edfcc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.688726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ec0b8-c229-41c5-8e62-f0946edfcc8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.689398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ec0b8-c229-41c5-8e62-f0946edfcc8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.691343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ec0b8-c229-41c5-8e62-f0946edfcc8b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.781217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 502/1007] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:30:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:20.580893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60f3e-dc2a-481d-a176-61f43beb8057 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.581866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60f3e-dc2a-481d-a176-61f43beb8057 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.582683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda60f3e-dc2a-481d-a176-61f43beb8057 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.585405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 506/1008] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:20.592232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844f8d-7cd0-4446-96af-b735f250216c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.592817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844f8d-7cd0-4446-96af-b735f250216c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.593294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844f8d-7cd0-4446-96af-b735f250216c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.622542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ea844f8d-7cd0-4446-96af-b735f250216c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:20.624756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ea844f8d-7cd0-4446-96af-b735f250216c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:20.691999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:20.729417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 503/1009] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:20.753253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60f3e-dc2a-481d-a176-61f43beb8057 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.863996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda60f3e-dc2a-481d-a176-61f43beb8057 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.904805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd7f3d-6fb3-4532-9b5e-211931bddd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.905473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd7f3d-6fb3-4532-9b5e-211931bddd98 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.905756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd7f3d-6fb3-4532-9b5e-211931bddd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.906033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd7f3d-6fb3-4532-9b5e-211931bddd98 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.906773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd7f3d-6fb3-4532-9b5e-211931bddd98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.908141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd7f3d-6fb3-4532-9b5e-211931bddd98 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.014411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 507/1010] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:20 2022] POST /identity/v3/services => generated 242 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:22.068902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485e6d4-f2bb-49bc-98eb-25cf86bc928f None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.068902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485e6d4-f2bb-49bc-98eb-25cf86bc928f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.068902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485e6d4-f2bb-49bc-98eb-25cf86bc928f None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.068902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 504/1011] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:22.069929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10803-828b-404c-9f05-140656cc8f74 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.069929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10803-828b-404c-9f05-140656cc8f74 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.069929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b10803-828b-404c-9f05-140656cc8f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.069929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c5b10803-828b-404c-9f05-140656cc8f74 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:22.069929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c5b10803-828b-404c-9f05-140656cc8f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.069929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.101480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 508/1012] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:22.125224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485e6d4-f2bb-49bc-98eb-25cf86bc928f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.217864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc5593-17e9-474e-af49-fe32002d32ec None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.218500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc5593-17e9-474e-af49-fe32002d32ec None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.219068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc5593-17e9-474e-af49-fe32002d32ec None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.224317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 509/1013] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:22.238372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.239271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.239874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.263097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485e6d4-f2bb-49bc-98eb-25cf86bc928f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.272231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:22.274376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.319254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.320721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.321376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.321868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.323240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.325854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:22.352058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:22.353803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:30:22.355174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:22.356776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 505/1014] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:22.365677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.403553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 510/1015] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:22 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:22.435566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.452944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.556157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.567883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efa3fa-c501-40eb-98ae-aedb7a67de4a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.594016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f0980-a93d-4d20-b182-745754f5ab20 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.627589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.628668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.628967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.629306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.631023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:22.631351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.637548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.649012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 506/1016] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:22.649378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.650661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.650828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.651127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.651707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.653246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.654911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.656719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 511/1017] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:22.663731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.790193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41c7cc-575a-40ef-b75d-eecd2f9d797a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.794853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.812268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfe006-4bdd-43d9-b475-75412a1c55af None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.832821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.833592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.833923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.834203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.834924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:22.835231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.836656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.847354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 507/1018] 10.222.0.58 () {56 vars in 1112 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:22.848375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.865238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930b8b9-60c5-49fb-bcf4-dda4cd218aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.866033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b8b9-60c5-49fb-bcf4-dda4cd218aa6 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.866397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b8b9-60c5-49fb-bcf4-dda4cd218aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.866755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930b8b9-60c5-49fb-bcf4-dda4cd218aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.867562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930b8b9-60c5-49fb-bcf4-dda4cd218aa6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.869231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930b8b9-60c5-49fb-bcf4-dda4cd218aa6 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.968797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.977950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 512/1019] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:22 2022] POST /identity/v3/projects => generated 311 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:23.013430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9165c-4bb8-44ac-be3d-06a72204b215 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:23.058746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55600f5-14c8-41c7-a579-2b3c52cdeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:23.059442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55600f5-14c8-41c7-a579-2b3c52cdeb87 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.059724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55600f5-14c8-41c7-a579-2b3c52cdeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.060005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55600f5-14c8-41c7-a579-2b3c52cdeb87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.060808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55600f5-14c8-41c7-a579-2b3c52cdeb87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:23.062236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55600f5-14c8-41c7-a579-2b3c52cdeb87 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:23.172569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 508/1020] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:30:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:23.419275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e53657-2da2-4016-a2c8-f7d1dd72bcbc None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.419804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e53657-2da2-4016-a2c8-f7d1dd72bcbc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.419804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e53657-2da2-4016-a2c8-f7d1dd72bcbc None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.421377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 513/1021] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:23.430311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.430616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.430913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.451587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:23.452896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:23.475683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:23.544163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 509/1022] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:23.568790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e53657-2da2-4016-a2c8-f7d1dd72bcbc None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:23.684186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e53657-2da2-4016-a2c8-f7d1dd72bcbc None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:23.718640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:23.719449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.719735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.720016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] PATH_INFO: `/v3/services/image` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.720711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:23.722629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:23.734674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:23.735690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:23.737395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 514/1023] 10.222.0.58 () {56 vars in 1105 bytes} [Fri May 27 03:30:23 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:23.742002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:23.850140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2e557-151f-4d37-8ac4-18b4819a0975 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:23.892899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c346342-42a0-4177-8375-74484753600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:23.894463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c346342-42a0-4177-8375-74484753600a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:23.895012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c346342-42a0-4177-8375-74484753600a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:23.895461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c346342-42a0-4177-8375-74484753600a None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:23.896337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c346342-42a0-4177-8375-74484753600a None admin] RBAC: Adding query filter params (name=image) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:23.896810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c346342-42a0-4177-8375-74484753600a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:23.898351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c346342-42a0-4177-8375-74484753600a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:23.908843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 510/1024] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:23.910563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:23.981987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:24.022112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbc72a-42ac-432c-870b-20be5be73c20 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:24.057548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:24.058606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:24.059106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:24.059605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:24.060549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] RBAC: Adding query filter params (type=image) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:24.061250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:24.062836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:24.069204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 515/1025] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:24.075386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c346342-42a0-4177-8375-74484753600a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:24.093513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:24.184478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c346342-42a0-4177-8375-74484753600a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:24.227230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:24.227895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:24.228176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:24.228450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:24.229272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] RBAC: Adding query filter params (interface=public, service_id=b82d3a9694374b4e8a3173a9d7e55a70, region_id=RegionOne) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:24.229585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:24.230957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c47beb-eb38-4e6e-9e96-7ff145c535a5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:24.237473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 511/1026] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:30:24 2022] GET /identity/v3/endpoints?service_id=b82d3a9694374b4e8a3173a9d7e55a70&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:24.241985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:24.353123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f6371-b96e-439f-a0d3-47f9f1a95dcc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:24.382706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:24.390980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec336e-5ad1-4698-bcf4-df9026f13241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:24.414354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec336e-5ad1-4698-bcf4-df9026f13241 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:24.414354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec336e-5ad1-4698-bcf4-df9026f13241 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:24.414354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec336e-5ad1-4698-bcf4-df9026f13241 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:24.414354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec336e-5ad1-4698-bcf4-df9026f13241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:24.414354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec336e-5ad1-4698-bcf4-df9026f13241 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:24.414354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 516/1027] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:30:24 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:25.415608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b37033-243f-4058-857e-98d4a4652b51 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.416542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b37033-243f-4058-857e-98d4a4652b51 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.416542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b37033-243f-4058-857e-98d4a4652b51 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.420615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 512/1028] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:25.423096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05db65-3dfc-4ceb-ae40-eb932eb6ae11 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.423526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05db65-3dfc-4ceb-ae40-eb932eb6ae11 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.423695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05db65-3dfc-4ceb-ae40-eb932eb6ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.441814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-dc05db65-3dfc-4ceb-ae40-eb932eb6ae11 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:25.443192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-dc05db65-3dfc-4ceb-ae40-eb932eb6ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:25.524439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 517/1029] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:25.548905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b37033-243f-4058-857e-98d4a4652b51 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:25.576361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9de5-0566-40aa-a594-746d6ddb419c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.576731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9de5-0566-40aa-a594-746d6ddb419c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.576955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9de5-0566-40aa-a594-746d6ddb419c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.578341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 518/1030] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:25.583717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.584005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.584287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.608606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:25.609793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:25.637573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:25.670848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b37033-243f-4058-857e-98d4a4652b51 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:25.687778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 519/1031] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:25.712108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:25.714770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:25.716444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.716753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.717030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.718229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:25.719678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:25.721999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 513/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:25.727379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:25.772840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:25.828422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117708e-009a-47a4-88b0-e3911fa6b0f1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:25.838613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae131e-ba0a-4552-9155-a6bf751058fc None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:25.866270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:25.867092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.870251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.870251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.883778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:25.884224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:25.884460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.884773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.885051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.885768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:25.887208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:25.893009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:25.894643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:25.896284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 520/1033] 10.222.0.58 () {56 vars in 1120 bytes} [Fri May 27 03:30:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:25.900411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:25.976087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:25.992533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 514/1034] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:30:25 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:26.015790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188aa04-d60a-4149-bc00-7854437a4f2f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.035265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:26.060128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920af781-732d-43ad-8e24-27273376faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.060834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920af781-732d-43ad-8e24-27273376faaf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.061124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920af781-732d-43ad-8e24-27273376faaf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.061399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920af781-732d-43ad-8e24-27273376faaf None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.062165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920af781-732d-43ad-8e24-27273376faaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:26.062476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920af781-732d-43ad-8e24-27273376faaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.064017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920af781-732d-43ad-8e24-27273376faaf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.074668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 521/1035] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:30:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:26.080252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.212332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e187f0f-db44-4d10-b530-245badb3e744 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.258004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.259057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.259512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.260859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.275108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.276676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:26.294493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:26.295474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:26.296421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:26.296421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:26.296421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:26.296421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:26.296421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 515/1036] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:26.302175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920af781-732d-43ad-8e24-27273376faaf None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.422406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920af781-732d-43ad-8e24-27273376faaf None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.470374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.472627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.473152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.473658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.474616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:26.475149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.476780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.490996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 522/1037] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:26.496463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.566089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:26.620976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c47ca-c50f-4002-b0a8-c20b11772e90 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.660739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.661618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.661816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.662085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.671583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.673545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-36291efd-4d61-4a9c-9dc9-24cc7eb3de7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:26.689246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:26.691153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:30:26.692625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:26.693508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 516/1038] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:26.697137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.697583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbe6ba-055d-4b30-8827-27a0fb63e9ce None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.697805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbe6ba-055d-4b30-8827-27a0fb63e9ce None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.698114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbe6ba-055d-4b30-8827-27a0fb63e9ce None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.699573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 517/1039] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:26.704649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.704946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.705222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.722566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:26.723759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:26.736281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:26.805268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 518/1040] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:26.818097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744be61-527c-4198-ba3a-1687cf64a26d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.828680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.852967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.853649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.853932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.854216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.855060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:26.855374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.856855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.886200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 523/1041] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:26.886720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.886854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:26.968092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51df96-a376-46f7-8a2a-52e1d3d609fe None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.995780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.005100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5936f6-db05-46fa-8273-44581d587afa None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.013817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.014742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.015118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.015484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] PATH_INFO: `/v3/services/image` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.016231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.017823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:27.031020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:27.032317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:30:27.033442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:27.035535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 519/1042] 10.222.0.58 () {56 vars in 1105 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/services/image => generated 86 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:27.041777 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.046341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.047183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.047567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.047940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.129303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.130668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.159361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59edd-ea0c-45d6-9966-769e16a01e49 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.169146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.203922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.204649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.204931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.205208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.206064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] RBAC: Adding query filter params (name=image) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:27.206691 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.208163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.215695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 520/1043] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:27.222602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.244447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.257015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 524/1044] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:27.263060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.368564 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36192ff1-7eea-47ff-be57-6eb8e4afab3e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.389036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7bd20-d52a-43a4-a5c2-e75744e0ef07 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.413452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.414126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.414440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.414689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.415423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:27.415732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.417307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.431197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 521/1045] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:27.432441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.435508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.436415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.436736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.437020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.448138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.449649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:27.468398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:27.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:27.471143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:27.471143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:27.471143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:27.471143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:27.475016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 525/1046] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:27.475677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.475962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.541153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.569296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e5991-0cd5-4a80-afd4-b09daf6e4de0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.607977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.608791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.609068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.609349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.610115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.611624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.616636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798e0c5-6b89-4c51-b57a-bb01a104cfae None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.657907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.658430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.658737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.659198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.660044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:27.660353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.661796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.676310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 526/1047] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:27.681617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.724234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 522/1048] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:27.803395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb26c8-0130-45ab-aeab-b434d1bcb0b1 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.806049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.857886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.858766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.859058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.859386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.870559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.870559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:27.887463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:27.888685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:27.889839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:27.891599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 527/1049] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:27.898187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.010969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.021606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d7816-1aae-414d-88a4-cdcda3f6ad05 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.059525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.059525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.059525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.059525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.060270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:28.062125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.064162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5036e85-ecd5-43da-b810-70758b3ca2bf None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.078614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 523/1050] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:28.085069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.206415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce47c3a-d565-4ced-9cf4-28aeb51345b9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.249585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.250471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.250854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.251232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.252229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] RBAC: Adding query filter params (role.id=ff0061288bd74b86a760a56c6f67ded2, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=0ceb759c7b4b4292bb9763574d79254b) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:28.252848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.255032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66920da6-2dcc-4d01-a7f9-4471c21fd93f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.268733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 528/1051] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/role_assignments?user.id=0ceb759c7b4b4292bb9763574d79254b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=ff0061288bd74b86a760a56c6f67ded2 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:28.364036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da4fb0-2362-4dc8-ac6d-fb81b818fd35 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.376353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da4fb0-2362-4dc8-ac6d-fb81b818fd35 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.376353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da4fb0-2362-4dc8-ac6d-fb81b818fd35 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.376353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 524/1052] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:30:28.376825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.376825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.376825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.398326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.399727 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.421624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.480548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 529/1053] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:28.500156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da4fb0-2362-4dc8-ac6d-fb81b818fd35 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.615519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da4fb0-2362-4dc8-ac6d-fb81b818fd35 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.652518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.653315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.653621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.653902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.664826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:28.672392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:28.673479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:28.674402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:28.676658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 525/1054] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:28.680035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.789897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5d980-6abb-4c47-a0cc-218fab25e3ef None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.829257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.830350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.831062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.831798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.832813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:28.833327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.834906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.844407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 530/1055] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:28.849797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.865730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.974990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a0593-56f7-400b-a06c-70340eac6906 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.013580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.014832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.015354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.015836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] PATH_INFO: `/v3/users/daffe1d6d6244cab8159048cc4619548` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.033561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] RBAC: Authorizing `identity:get_user(user_id=daffe1d6d6244cab8159048cc4619548)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.035763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.039935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 526/1056] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/users/daffe1d6d6244cab8159048cc4619548 => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:29.043360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.063152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.157181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2a7df-de53-4814-a539-7a6bb4bfebe4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.193988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.195019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.195517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.195994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.272740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.274589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.292590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.392545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 531/1057] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:29.395568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.469637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.508979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13e14-c750-4475-8cbc-2c2f2f0e660e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.551860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5383b20-5e51-4cbe-945e-276f517062d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.552781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383b20-5e51-4cbe-945e-276f517062d3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.553063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383b20-5e51-4cbe-945e-276f517062d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.553337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383b20-5e51-4cbe-945e-276f517062d3 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.554630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5383b20-5e51-4cbe-945e-276f517062d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d59f3442b7e64265bc967f79dcaf7edd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.556548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5383b20-5e51-4cbe-945e-276f517062d3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.558537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 527/1058] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:29.564514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.668283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc9cf4-4441-411b-9571-e0319a1d21d9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.688445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.707862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.708528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.708825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.709101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.709932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=d59f3442b7e64265bc967f79dcaf7edd, user.id=daffe1d6d6244cab8159048cc4619548) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:29.710243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.711855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fcfd8-569a-4da3-be12-137cba03b4b8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.724628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 532/1059] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/role_assignments?user.id=daffe1d6d6244cab8159048cc4619548&scope.project.id=d59f3442b7e64265bc967f79dcaf7edd&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:30.153701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e9ec-74d0-4921-8397-23ca6e542b28 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.154212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e9ec-74d0-4921-8397-23ca6e542b28 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.154212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e9ec-74d0-4921-8397-23ca6e542b28 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.155691 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 528/1060] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:30.161772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6f1c-0e7b-4dff-a583-2f643d78dccd None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.162068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6f1c-0e7b-4dff-a583-2f643d78dccd None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.162393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6f1c-0e7b-4dff-a583-2f643d78dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.179342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-848a6f1c-0e7b-4dff-a583-2f643d78dccd None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:30.180585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-848a6f1c-0e7b-4dff-a583-2f643d78dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.260943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 533/1061] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:30.283691 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e9ec-74d0-4921-8397-23ca6e542b28 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.397725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e9ec-74d0-4921-8397-23ca6e542b28 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.437416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.438098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.438382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.438659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.443022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.517190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.519893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.523251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ee12-f0b1-441e-9bb8-f3b53a323d33 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.523551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ee12-f0b1-441e-9bb8-f3b53a323d33 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.523837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ee12-f0b1-441e-9bb8-f3b53a323d33 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.525579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 534/1062] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:30:30.531922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.531922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.531922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.550589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:30.552073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.641998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 529/1063] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:30.642535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 535/1064] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:30.647830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.655161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.693300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.694239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.762426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ee11-f367-4f84-a76a-0d40d82e2e1f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.765552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff66b5-bb89-449e-961f-f4fc01ded78c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.801455 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e74669-93a3-402a-9928-0e509884212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.802417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e74669-93a3-402a-9928-0e509884212b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.802706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e74669-93a3-402a-9928-0e509884212b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.802988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e74669-93a3-402a-9928-0e509884212b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.805934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.806748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.807041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.807320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.813585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e74669-93a3-402a-9928-0e509884212b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.815900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e74669-93a3-402a-9928-0e509884212b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.819195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:30.827380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:30.828442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:30.829350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 530/1065] 10.222.0.58 () {56 vars in 1120 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:30.836373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-02e74669-93a3-402a-9928-0e509884212b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:30.839859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:30.840826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:30.841788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:30.841788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:30.841788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:30.841788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:30.841788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 536/1066] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:30.847267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e74669-93a3-402a-9928-0e509884212b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.957815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.963916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.967576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e74669-93a3-402a-9928-0e509884212b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.970174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f6682-cd69-406a-b25a-ba6fcc877783 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.007775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.008302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.008529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.008814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.008951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.019228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.019382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.019382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.019538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:31.019538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.019714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:31.019714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.019869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.019949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.024530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 531/1067] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:31.028544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 537/1068] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:31.031060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.156770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08efe0f-3d43-44b5-bcd9-7d42d6c9ae15 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.197378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.198210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.198495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.198771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.211687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.213111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:31.231348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:31.232313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:31.233460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:31.233460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:31.233460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:31.233460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:31.233460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 532/1069] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:31.239545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.359743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15914aae-9e32-4941-8418-44e1872a12de None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.399615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.400286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.400596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.400879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.401602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:31.401913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.403288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.417747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 538/1070] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:31.423295 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.438055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:31.549428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9c4b3-6c29-4ee1-9d55-d3d4ec9001e9 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.574665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.576035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.576476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.576921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.587495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.589858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:31.606461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:31.607538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:31.608544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 533/1071] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:31.615766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.632228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:31.729413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51efb4-2886-48d8-b79d-114d59ff2cd8 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.767770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.769060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.769348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.769663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.770475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:31.770783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.772231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.794729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 539/1072] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:31.800975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.886431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:31.921933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda836a-1590-4880-a7f8-ffcbaf857087 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.961025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.961837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.962126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.962404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.974457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.975949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8a96fd05-85eb-499f-ad1e-43fc34830732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:31.995351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:31.997755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:32.000224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:32.000224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:32.000224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:32.000224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:32.000224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 534/1073] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:32.003891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.017606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adb8ac-4a90-42b1-b8fe-3ad9ce1c8678 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.017890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adb8ac-4a90-42b1-b8fe-3ad9ce1c8678 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.018440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adb8ac-4a90-42b1-b8fe-3ad9ce1c8678 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.019920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 535/1074] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:32.025962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.026254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.026537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.045036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:32.046258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:32.091613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.130943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df5207-30c1-4d0d-b40a-5ba017880c2b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.136660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 536/1075] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:30:32.153076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.176097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.177063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.177358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.177645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.178394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:32.178705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.180097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.193096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 540/1076] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:32.202255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.217252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.282966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ecc6b0-8f41-450b-80f4-3386f704a26d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.331494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f7307-3f09-42f4-a501-28db1686c514 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.334155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.335355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.335792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.336324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.352238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:32.359832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:32.360939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:32.364009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 537/1077] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:32.368203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.368372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.375869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.376952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.377452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.377801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.386725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.388413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:32.406940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:32.410437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:32.411374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:32.412686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 541/1078] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:32.414657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.491834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a74d99-0601-4abb-bda2-7886ed1ce088 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.508230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.540003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.540843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.541323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.541516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.542812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:32.543164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.544797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.550506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e662a-7c3e-48ec-871a-4001e6c177b4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.561101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 538/1079] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:32.564290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.593363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.594051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.594339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.594625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.595438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:32.595752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.597827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.614378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 542/1080] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:32.620862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.688915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.703487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b39f69-1f92-4e74-aa64-cdfaf69373f8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.745503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.746453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.746746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.747020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] PATH_INFO: `/v3/users/daffe1d6d6244cab8159048cc4619548` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.748233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] RBAC: Authorizing `identity:get_user(user_id=daffe1d6d6244cab8159048cc4619548)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.748451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890b755-8c02-4890-9b1a-8be023dcfe62 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.750043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.752546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 539/1081] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/users/daffe1d6d6244cab8159048cc4619548 => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:32.757858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.790557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.791537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.791822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.792097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/0ceb759c7b4b4292bb9763574d79254b/roles/ff0061288bd74b86a760a56c6f67ded2` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.794410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=0ceb759c7b4b4292bb9763574d79254b, role_id=ff0061288bd74b86a760a56c6f67ded2)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.795945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.806533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.866348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af92606-20b2-42ca-b6a2-4b09f7b5ab30 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.888552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 543/1082] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:30:32 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/0ceb759c7b4b4292bb9763574d79254b/roles/ff0061288bd74b86a760a56c6f67ded2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:32.910330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371ea00-a221-422e-8f8c-07253f97fb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.911242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ea00-a221-422e-8f8c-07253f97fb86 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.911520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ea00-a221-422e-8f8c-07253f97fb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.911791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371ea00-a221-422e-8f8c-07253f97fb86 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.913280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371ea00-a221-422e-8f8c-07253f97fb86 None admin] RBAC: Authorizing `identity:get_project(project_id=d59f3442b7e64265bc967f79dcaf7edd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.914964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371ea00-a221-422e-8f8c-07253f97fb86 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.917268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 540/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:32.923580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.056086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b89c7-f3d0-464c-95e0-3f045ab5217b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.069567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:33.101123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d07ee-d940-43ac-9d93-cc7cd8621b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.102044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d07ee-d940-43ac-9d93-cc7cd8621b54 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.102335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d07ee-d940-43ac-9d93-cc7cd8621b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.102619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d07ee-d940-43ac-9d93-cc7cd8621b54 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd/users/daffe1d6d6244cab8159048cc4619548/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.105942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d07ee-d940-43ac-9d93-cc7cd8621b54 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59f3442b7e64265bc967f79dcaf7edd, user_id=daffe1d6d6244cab8159048cc4619548, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.108550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d07ee-d940-43ac-9d93-cc7cd8621b54 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.255268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 544/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:30:32 2022] PUT /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd/users/daffe1d6d6244cab8159048cc4619548/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:35.158276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97ce4-1dc7-4ee8-87df-61157791ca83 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.159258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97ce4-1dc7-4ee8-87df-61157791ca83 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.159340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e97ce4-1dc7-4ee8-87df-61157791ca83 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.161036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 541/1085] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:35.167206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efff197-5313-4fae-9837-9d249341ad2b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.167743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efff197-5313-4fae-9837-9d249341ad2b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.167743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efff197-5313-4fae-9837-9d249341ad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.193700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-3efff197-5313-4fae-9837-9d249341ad2b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:35.194908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3efff197-5313-4fae-9837-9d249341ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:35.289360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 545/1086] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:35.317275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e97ce4-1dc7-4ee8-87df-61157791ca83 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:35.541412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e97ce4-1dc7-4ee8-87df-61157791ca83 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:35.554559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:35.580368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:35.582027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.582273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.582546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.597426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:35.604097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e980496-97ef-4074-a6ae-2682b17debf1 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.604382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e980496-97ef-4074-a6ae-2682b17debf1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.605001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e980496-97ef-4074-a6ae-2682b17debf1 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.606539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 546/1087] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:35.610718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:35.611653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:35.613180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 542/1088] 10.222.0.58 () {56 vars in 1120 bytes} [Fri May 27 03:30:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:35.613876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.613876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.613876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.618418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:35.636181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:35.637740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:35.732318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 547/1089] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:35.757605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde4ee-8aac-45a7-a74e-38ed985f084b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:35.757955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:35.802985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:35.803937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.804375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.804838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.804976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:35.805609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:35.805925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:35.807330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:35.817560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 543/1090] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:30:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:35.825723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:35.882667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:35.896893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e111e89-837d-4c69-a0ba-f0dddadb3a2e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:35.947906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:35.949204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:35.949678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:35.950145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:35.964044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e1824-5507-4051-8edd-bdf2e3901f1e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:35.964735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:35.974883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:35.976112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:35.977087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:35.979232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 548/1091] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:35.986042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.011526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.012323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.012633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.012907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.025176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.027763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:36.050160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:36.051335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:36.052264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:36.052264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:36.052264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:36.052264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:36.054698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 544/1092] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:36.060439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.118452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.132204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee701a-f177-46f3-bc2e-77c39f3811f3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.176214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.180187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.180908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.181193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.181471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.182230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:36.182540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.183928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.194857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69392144-9281-4f46-b939-705218b3daa3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.195372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 549/1093] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:36.201630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.247917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.248624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.248921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.249429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.250164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:36.250474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.251945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.266346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 545/1094] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:36.273509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.334282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6821d-0ffe-4fe9-98d9-9e9c4af62a70 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.387692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.390261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.390261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.390261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] PATH_INFO: `/v3/users/daffe1d6d6244cab8159048cc4619548` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.390991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] RBAC: Authorizing `identity:get_user(user_id=daffe1d6d6244cab8159048cc4619548)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.392665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.394633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 550/1095] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/users/daffe1d6d6244cab8159048cc4619548 => generated 285 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:36.400586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.408228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67840a9c-7460-41b3-ad87-08038797ff8b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.425112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.452682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.453655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.454039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.454203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.465196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.466695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:30:36.483057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:36.484268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:30:36.485550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:36.487944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 546/1096] 10.222.0.58 () {56 vars in 1093 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:36.494078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.499331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.523704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683de1f6-8eb8-456c-9b93-39a5f30d6767 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.570265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.571912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.572449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.572938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.619914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cabed-1df8-414f-932d-08833f841545 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.662276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.662962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.663435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.663717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.664766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:36.665285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.666993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.670385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.672045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:36.696314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 547/1097] 10.222.0.58 () {56 vars in 1133 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:36.697957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.702742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.780090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.807460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 551/1098] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:30:36.813627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:36.848805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622e75-d15e-438e-b152-8bd19d8e0406 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.898140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:36.899270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:36.899640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:36.899984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:36.965519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704a0a8-6022-4dcc-96b0-ed744761e659 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:36.991240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:36.993194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:36.994764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.011852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.014862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.014862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.014862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.015606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] RBAC: Authorizing `identity:get_project(project_id=d59f3442b7e64265bc967f79dcaf7edd)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.017204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.019530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 552/1099] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd => generated 311 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:37.031756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.068816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.128730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 548/1100] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:30:37.135733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.175407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8bd2d-ffc8-4078-9de4-2b6cf313a461 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.228566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.229355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.229467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.229906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.230871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=d59f3442b7e64265bc967f79dcaf7edd, user.id=daffe1d6d6244cab8159048cc4619548) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:37.231332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.232973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.240693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 553/1101] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/role_assignments?user.id=daffe1d6d6244cab8159048cc4619548&scope.project.id=d59f3442b7e64265bc967f79dcaf7edd&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:37.267707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f73a1-7549-42d6-ab06-22c0903c95d0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.323576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.324744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.325211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.325657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.340875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.343257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:30:37.370988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:37.373920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:37.375580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:30:37.375580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:30:37.375580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:30:37.375580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:37.375580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 549/1102] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:30:37.380606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.456524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.527594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf6e0c-2f95-4de5-a6e6-718ff6f1fb6e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.583108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.584154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.584672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.585177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.586111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:37.586619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.588280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.607221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 554/1103] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:37.609397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.630657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:37.753411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c594023-fdaa-4462-a7bd-4bcac6f29d2a None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:37.803213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:37.804046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:37.804328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:37.804903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:37.813570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:37.815661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:30:37.831407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:37.832410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:30:37.833379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:30:37.841318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 550/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:37.841973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:37.984284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81cd83-6f97-4e61-84c7-04d6de60ba24 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.000560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:38.037925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.039302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.039818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.040275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.041366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:38.041694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.043170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d01e11-7512-4d63-98c9-f5e857a64eab None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.060834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 555/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:38.068773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:38.182674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:38.219385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a844d-1a97-48e8-8a40-334907b16015 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:38.270870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:38.272038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:38.272038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:38.272038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:38.272956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] RBAC: Adding query filter params (role.id=ff0061288bd74b86a760a56c6f67ded2, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=0ceb759c7b4b4292bb9763574d79254b) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:38.273298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:38.274756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44048989-a610-4ed0-b284-3663278cd051 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:38.281582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 551/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:38 2022] GET /identity/v3/role_assignments?user.id=0ceb759c7b4b4292bb9763574d79254b&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=ff0061288bd74b86a760a56c6f67ded2 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:39.732107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eab795-09cd-4c7e-b345-294e8268942e None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:39.732895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eab795-09cd-4c7e-b345-294e8268942e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:39.732895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eab795-09cd-4c7e-b345-294e8268942e None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:39.734142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 556/1107] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:39.741155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:39.741712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:39.741712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:39.760667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:39.761717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:39.840755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 552/1108] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:39.866047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eab795-09cd-4c7e-b345-294e8268942e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:40.004226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eab795-09cd-4c7e-b345-294e8268942e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:40.040258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb972e2-c881-4009-b17c-32a5860902df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:40.041096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb972e2-c881-4009-b17c-32a5860902df None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:40.041382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb972e2-c881-4009-b17c-32a5860902df None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:40.041669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb972e2-c881-4009-b17c-32a5860902df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:40.042868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb972e2-c881-4009-b17c-32a5860902df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:40.044337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb972e2-c881-4009-b17c-32a5860902df None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:40.046039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 557/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:40.053560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:40.128289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:40.173692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de874d3-e009-4a4c-a4b3-d1872d6b805e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:40.227102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3618a44-87c5-4504-92b9-4eeed14a998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:40.227803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3618a44-87c5-4504-92b9-4eeed14a998c None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:40.228083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3618a44-87c5-4504-92b9-4eeed14a998c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:40.228361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3618a44-87c5-4504-92b9-4eeed14a998c None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:40.229143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3618a44-87c5-4504-92b9-4eeed14a998c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:40.230670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3618a44-87c5-4504-92b9-4eeed14a998c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:40.351776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 553/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:30:40 2022] POST /identity/v3/users => generated 286 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:42.691874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0640f01-ee28-46e8-b8ef-a62abd47dce3 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:42.692542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0640f01-ee28-46e8-b8ef-a62abd47dce3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:42.692542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0640f01-ee28-46e8-b8ef-a62abd47dce3 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:42.693928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 558/1111] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:42.700061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:42.700644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:42.700644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:42.719345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:42.720617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:42.793696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 554/1112] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:42.818088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0640f01-ee28-46e8-b8ef-a62abd47dce3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:42.863192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:42.930703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0640f01-ee28-46e8-b8ef-a62abd47dce3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:42.967121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:42.967902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:42.968187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:42.968465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] PATH_INFO: `/v3/roles/c636e98b844543518e09f390c4850083` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:42.969995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] RBAC: Authorizing `identity:get_role(role_id=c636e98b844543518e09f390c4850083)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:42.971595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:42.973320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 559/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:42 2022] GET /identity/v3/roles/c636e98b844543518e09f390c4850083 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:42.979229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:43.044826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:43.092180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afb0ba-3f0c-41c8-90c4-de787a7a303e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:43.129359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:43.130244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:43.130550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:43.130831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] PATH_INFO: `/v3/users/833a3469e0b448d789ea1cb0b5de4743` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:43.145684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] RBAC: Authorizing `identity:get_user(user_id=833a3469e0b448d789ea1cb0b5de4743)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:43.147146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:43.148909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 555/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:42 2022] GET /identity/v3/users/833a3469e0b448d789ea1cb0b5de4743 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:43.154871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:43.263957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165fe9e-14a6-4475-b7bd-955b8d7f917f None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:43.273689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:43.298809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:43.300590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:43.300895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:43.301174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:43.380312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:43.381805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:43.483172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 560/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:43.491362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:43.594180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc9d58-d2a1-4e7f-95cc-d4bd2ed8a898 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:43.607939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:43.639984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197b6f1-4721-4dc6-a8a5-d27579aafcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:43.640926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197b6f1-4721-4dc6-a8a5-d27579aafcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:43.641205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197b6f1-4721-4dc6-a8a5-d27579aafcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:43.641480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197b6f1-4721-4dc6-a8a5-d27579aafcc5 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:43.642656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197b6f1-4721-4dc6-a8a5-d27579aafcc5 None admin] RBAC: Authorizing `identity:get_project(project_id=d59f3442b7e64265bc967f79dcaf7edd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:43.644804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197b6f1-4721-4dc6-a8a5-d27579aafcc5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:43.646661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 556/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:43 2022] GET /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:43.652883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:43.664389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:43.759986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a7457-fa7e-4c28-8d23-cd2cc96a05c9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:43.797113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:43.797764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:43.798234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:43.798234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:43.799152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] RBAC: Adding query filter params (role.id=c636e98b844543518e09f390c4850083, scope.project.id=d59f3442b7e64265bc967f79dcaf7edd, user.id=833a3469e0b448d789ea1cb0b5de4743) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:43.800016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:43.801496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b076c0-b9e3-4943-9049-b16d720dc978 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:43.807071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 561/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:43 2022] GET /identity/v3/role_assignments?user.id=833a3469e0b448d789ea1cb0b5de4743&scope.project.id=d59f3442b7e64265bc967f79dcaf7edd&role.id=c636e98b844543518e09f390c4850083 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:46.040598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3933318-d471-4cdd-8196-8ed63b8a3a56 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:46.041126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3933318-d471-4cdd-8196-8ed63b8a3a56 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:46.041238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3933318-d471-4cdd-8196-8ed63b8a3a56 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:46.042517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 557/1118] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:46.048035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:46.048466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:46.048616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:46.065420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:46.066606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:46.135419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 562/1119] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:46.159580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3933318-d471-4cdd-8196-8ed63b8a3a56 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:46.267857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3933318-d471-4cdd-8196-8ed63b8a3a56 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:46.301840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:46.302692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:46.302977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:46.303252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] PATH_INFO: `/v3/roles/c636e98b844543518e09f390c4850083` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:46.304746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] RBAC: Authorizing `identity:get_role(role_id=c636e98b844543518e09f390c4850083)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:46.306122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:46.307494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 558/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:46 2022] GET /identity/v3/roles/c636e98b844543518e09f390c4850083 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:46.313984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:46.317017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:46.416374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9d758-270a-498b-8219-219d4a5e3ed3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:46.450576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:46.451456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:46.451835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:46.452037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] PATH_INFO: `/v3/users/833a3469e0b448d789ea1cb0b5de4743` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:46.453333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] RBAC: Authorizing `identity:get_user(user_id=833a3469e0b448d789ea1cb0b5de4743)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:46.454776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:46.457005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 563/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:46 2022] GET /identity/v3/users/833a3469e0b448d789ea1cb0b5de4743 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:46.463339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:46.513940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:46.565114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40cdd-4f68-45b4-9474-7f648ded2dbb None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:46.598712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366e160-6c19-43fb-af62-a9746ac31c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:46.599557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e160-6c19-43fb-af62-a9746ac31c0f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:46.599844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e160-6c19-43fb-af62-a9746ac31c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:46.600125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366e160-6c19-43fb-af62-a9746ac31c0f None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:46.601322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366e160-6c19-43fb-af62-a9746ac31c0f None admin] RBAC: Authorizing `identity:get_project(project_id=d59f3442b7e64265bc967f79dcaf7edd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:46.602767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366e160-6c19-43fb-af62-a9746ac31c0f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:46.604508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 559/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:46 2022] GET /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:46.610554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:46.679392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:46.708461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe43d55-fa4d-4eb3-9528-8377f35dc1aa None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:46.742786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035163bd-7747-40e5-a271-d7cad3411606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:46.743664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035163bd-7747-40e5-a271-d7cad3411606 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:46.743953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035163bd-7747-40e5-a271-d7cad3411606 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:46.744235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035163bd-7747-40e5-a271-d7cad3411606 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd/users/833a3469e0b448d789ea1cb0b5de4743/roles/c636e98b844543518e09f390c4850083` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:46.746464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035163bd-7747-40e5-a271-d7cad3411606 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59f3442b7e64265bc967f79dcaf7edd, user_id=833a3469e0b448d789ea1cb0b5de4743, role_id=c636e98b844543518e09f390c4850083)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:46.747898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035163bd-7747-40e5-a271-d7cad3411606 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:46.813442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 564/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:30:46 2022] PUT /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd/users/833a3469e0b448d789ea1cb0b5de4743/roles/c636e98b844543518e09f390c4850083 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:49.027531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb98189-df1e-4252-a62a-820ebce21f38 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:49.028095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb98189-df1e-4252-a62a-820ebce21f38 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:49.028095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb98189-df1e-4252-a62a-820ebce21f38 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:49.029486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 560/1124] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:49.034864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:49.035299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:49.035425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:49.052680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:49.053935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:49.122490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 565/1125] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:49.147572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb98189-df1e-4252-a62a-820ebce21f38 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:49.248750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb98189-df1e-4252-a62a-820ebce21f38 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:49.265781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:49.283523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:49.285893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:49.285893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:49.285893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] PATH_INFO: `/v3/roles/c636e98b844543518e09f390c4850083` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:49.287529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] RBAC: Authorizing `identity:get_role(role_id=c636e98b844543518e09f390c4850083)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:49.288942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:49.290347 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 561/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:49 2022] GET /identity/v3/roles/c636e98b844543518e09f390c4850083 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:49.296246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:49.354737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:49.394755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026eda10-87f2-4002-a4e1-0338e21244e4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:49.429256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:49.430130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:49.430417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:49.430697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] PATH_INFO: `/v3/users/833a3469e0b448d789ea1cb0b5de4743` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:49.431879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] RBAC: Authorizing `identity:get_user(user_id=833a3469e0b448d789ea1cb0b5de4743)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:49.433350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:49.434988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 566/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:49 2022] GET /identity/v3/users/833a3469e0b448d789ea1cb0b5de4743 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:49.440872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:49.540869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2d622-e899-486a-a913-102187a626f6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:49.575675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:49.576324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:49.576642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:49.576926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:49.628515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:49.643623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:49.645344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:49.744994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 562/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:49.751139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:49.850297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11705f9e-532d-4e0b-a506-be12d93c649e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:49.876473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:49.884398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d19f5e-26ae-490e-a00c-6d941a6e1260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:49.885379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d19f5e-26ae-490e-a00c-6d941a6e1260 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:49.885596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d19f5e-26ae-490e-a00c-6d941a6e1260 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:49.885963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d19f5e-26ae-490e-a00c-6d941a6e1260 None admin] PATH_INFO: `/v3/projects/d59f3442b7e64265bc967f79dcaf7edd` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:49.887161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d19f5e-26ae-490e-a00c-6d941a6e1260 None admin] RBAC: Authorizing `identity:get_project(project_id=d59f3442b7e64265bc967f79dcaf7edd)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:49.888627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d19f5e-26ae-490e-a00c-6d941a6e1260 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:49.890514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 567/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:49 2022] GET /identity/v3/projects/d59f3442b7e64265bc967f79dcaf7edd => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:49.897303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:49.985926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:50.002459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ed063-288f-4317-a0ea-6cb946625cba None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:50.039662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:50.040339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:50.040668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:50.040949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:50.041778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] RBAC: Adding query filter params (role.id=c636e98b844543518e09f390c4850083, scope.project.id=d59f3442b7e64265bc967f79dcaf7edd, user.id=833a3469e0b448d789ea1cb0b5de4743) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:50.042094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:50.043537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823b6a-59e8-4f3c-9b63-2b9da39c4564 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:50.049558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 563/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:49 2022] GET /identity/v3/role_assignments?user.id=833a3469e0b448d789ea1cb0b5de4743&scope.project.id=d59f3442b7e64265bc967f79dcaf7edd&role.id=c636e98b844543518e09f390c4850083 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:52.295999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad28805-6379-4adb-98a5-387a551d4b29 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:52.296584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad28805-6379-4adb-98a5-387a551d4b29 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:52.296584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad28805-6379-4adb-98a5-387a551d4b29 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:52.297972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 568/1131] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:52.303935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:52.304355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:52.304514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:52.322151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:52.323361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:52.392361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 564/1132] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:52.416693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad28805-6379-4adb-98a5-387a551d4b29 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:52.519878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad28805-6379-4adb-98a5-387a551d4b29 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:52.556689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f0876-9390-4218-9ce3-118c6849073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:52.557404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f0876-9390-4218-9ce3-118c6849073f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:52.557701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f0876-9390-4218-9ce3-118c6849073f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:52.557981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f0876-9390-4218-9ce3-118c6849073f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:52.559187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f0876-9390-4218-9ce3-118c6849073f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:52.560673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f0876-9390-4218-9ce3-118c6849073f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:52.562316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 569/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:52.568204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:52.644691 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:52.665704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac851-6fc7-47d1-b22f-c07ef69a5f15 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:52.700392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8aaa6-e140-43b9-9b75-a7925c42d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:52.701073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8aaa6-e140-43b9-9b75-a7925c42d83c None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:52.701360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8aaa6-e140-43b9-9b75-a7925c42d83c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:52.702401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8aaa6-e140-43b9-9b75-a7925c42d83c None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:52.702401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8aaa6-e140-43b9-9b75-a7925c42d83c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:52.703882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8aaa6-e140-43b9-9b75-a7925c42d83c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:52.797328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 565/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:30:52 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:55.103804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990447bd-cac2-416c-a1c4-f5e0a5ea02f5 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.104351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990447bd-cac2-416c-a1c4-f5e0a5ea02f5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.104351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990447bd-cac2-416c-a1c4-f5e0a5ea02f5 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.105767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 570/1135] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:55.111093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.111469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.111657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.130223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:55.131344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:55.198255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 566/1136] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:55.222854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990447bd-cac2-416c-a1c4-f5e0a5ea02f5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.287847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:55.325103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990447bd-cac2-416c-a1c4-f5e0a5ea02f5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.359449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ade57-d121-4860-ba21-8ebd5e4f0fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.360227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ade57-d121-4860-ba21-8ebd5e4f0fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.360527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ade57-d121-4860-ba21-8ebd5e4f0fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.360807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ade57-d121-4860-ba21-8ebd5e4f0fc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.361973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ade57-d121-4860-ba21-8ebd5e4f0fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.363457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ade57-d121-4860-ba21-8ebd5e4f0fc8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.365092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 571/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:30:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:55.371135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.432868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:55.474956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4672cab-2ea7-4e7b-9015-c4a89a51129b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.509540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1628f-6f6d-4dc0-af21-14b3242e9eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.510207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1628f-6f6d-4dc0-af21-14b3242e9eab None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.510492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1628f-6f6d-4dc0-af21-14b3242e9eab None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.510765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1628f-6f6d-4dc0-af21-14b3242e9eab None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.511466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1628f-6f6d-4dc0-af21-14b3242e9eab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.512967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1628f-6f6d-4dc0-af21-14b3242e9eab None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.596745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 567/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:57.829922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884bba-8c7d-4e92-94b8-5b69a3447e9c None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.830505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884bba-8c7d-4e92-94b8-5b69a3447e9c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.830505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e884bba-8c7d-4e92-94b8-5b69a3447e9c None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.832145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 572/1139] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:30:57.837326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.837733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.837883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.855963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:57.857186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:57.927177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 568/1140] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:30:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:30:57.950167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884bba-8c7d-4e92-94b8-5b69a3447e9c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.051119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e884bba-8c7d-4e92-94b8-5b69a3447e9c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.079680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.085451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.086184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.086468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.086764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.096731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:30:58.104999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:30:58.106197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:30:58.107272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 573/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:30:58.112240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.161389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.216887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51fd6a-e602-4638-95d3-90ef995d5c9a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.252594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.253855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.254337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.254637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.255390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:58.255701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.257113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.265846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 569/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:58.272063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.367803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316ab4e-8620-4ff7-8d70-9ce883565781 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.400467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.401359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.401639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.401913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] PATH_INFO: `/v3/users/4a4e10c26ca649dbbeb3afc60d06f830` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.415394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] RBAC: Authorizing `identity:get_user(user_id=4a4e10c26ca649dbbeb3afc60d06f830)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.416850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.418495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 574/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:30:58 2022] GET /identity/v3/users/4a4e10c26ca649dbbeb3afc60d06f830 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:58.424469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.528523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f11f16-c9fe-4b97-a404-44f55481507c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.538470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.567184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.567849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.568180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.568458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.636531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.637921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.736600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 570/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:30:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:30:58.743246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.793559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.841445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d7fe3-c7b7-4bd8-a811-54e2a71e71bb None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.880776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185ff06-2db5-425c-ad3b-ded93a975ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.881662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185ff06-2db5-425c-ad3b-ded93a975ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.881946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185ff06-2db5-425c-ad3b-ded93a975ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.882225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185ff06-2db5-425c-ad3b-ded93a975ad4 None admin] PATH_INFO: `/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.883434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185ff06-2db5-425c-ad3b-ded93a975ad4 None admin] RBAC: Authorizing `identity:get_project(project_id=2919ac5ecfeb449bbbef9f55b85ae5d3)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.884901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185ff06-2db5-425c-ad3b-ded93a975ad4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.886734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 575/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:30:58 2022] GET /identity/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:58.893105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.903812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.998613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c54c2-1ec4-4f80-ab18-81a6b9a8c64f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.032779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.033438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.033729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.034011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.035385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=2919ac5ecfeb449bbbef9f55b85ae5d3, user.id=4a4e10c26ca649dbbeb3afc60d06f830) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:59.035696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.037206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7748aa-6ecc-4c2b-b92a-85acfbcdbc3c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.043068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 571/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:30:58 2022] GET /identity/v3/role_assignments?user.id=4a4e10c26ca649dbbeb3afc60d06f830&scope.project.id=2919ac5ecfeb449bbbef9f55b85ae5d3&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:01.261047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29c00e-3ba5-4989-9eef-9432b6aede93 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.261609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29c00e-3ba5-4989-9eef-9432b6aede93 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.261609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29c00e-3ba5-4989-9eef-9432b6aede93 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.262959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 576/1147] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:01.268398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.269331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.269536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.286770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:01.287973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:01.353440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 572/1148] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:01.376733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29c00e-3ba5-4989-9eef-9432b6aede93 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:01.477094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29c00e-3ba5-4989-9eef-9432b6aede93 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:01.495012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:01.514146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:01.514146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.514146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.514146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.524688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:31:01.532298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:31:01.534166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:01.536314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:31:01.538895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 577/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:31:01.539989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:01.542275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:01.640615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1418308-df36-43cc-bb6e-6d73fae1c8e5 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:01.673489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:01.674156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.674436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.674709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.675530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:01.675837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:01.677219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:01.685068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 573/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:31:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:01.691564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:01.790818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a211de-7b69-492d-a653-7d6ec13d765e None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:01.823725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:01.824619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.824910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.825189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] PATH_INFO: `/v3/users/4a4e10c26ca649dbbeb3afc60d06f830` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.826386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] RBAC: Authorizing `identity:get_user(user_id=4a4e10c26ca649dbbeb3afc60d06f830)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:01.827812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:01.829526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 578/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:31:01 2022] GET /identity/v3/users/4a4e10c26ca649dbbeb3afc60d06f830 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:01.835888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:01.898801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:01.937402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9784c-9cab-4e2a-a3d0-c7c7842257e6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:01.972842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade2e2d-926e-442a-93f3-0de325761c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:01.973704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2e2d-926e-442a-93f3-0de325761c62 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.973987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2e2d-926e-442a-93f3-0de325761c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.974263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade2e2d-926e-442a-93f3-0de325761c62 None admin] PATH_INFO: `/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.975441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade2e2d-926e-442a-93f3-0de325761c62 None admin] RBAC: Authorizing `identity:get_project(project_id=2919ac5ecfeb449bbbef9f55b85ae5d3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:01.976908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade2e2d-926e-442a-93f3-0de325761c62 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:01.978599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 574/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:31:01 2022] GET /identity/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:01.984686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:02.030633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:02.083718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01231f16-aedb-4af4-830c-12103971ba9c None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:02.118481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5beae0-d6f0-441f-aee0-c1abe60c3197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:02.119373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5beae0-d6f0-441f-aee0-c1abe60c3197 None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:02.119658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5beae0-d6f0-441f-aee0-c1abe60c3197 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:02.119932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5beae0-d6f0-441f-aee0-c1abe60c3197 None admin] PATH_INFO: `/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3/users/4a4e10c26ca649dbbeb3afc60d06f830/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:02.121928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5beae0-d6f0-441f-aee0-c1abe60c3197 None admin] RBAC: Authorizing `identity:create_grant(project_id=2919ac5ecfeb449bbbef9f55b85ae5d3, user_id=4a4e10c26ca649dbbeb3afc60d06f830, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:02.123367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5beae0-d6f0-441f-aee0-c1abe60c3197 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:02.219469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 579/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:31:01 2022] PUT /identity/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3/users/4a4e10c26ca649dbbeb3afc60d06f830/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:04.399737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee7a4c-84b9-46c9-b0f9-e83f9e25d398 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.400294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee7a4c-84b9-46c9-b0f9-e83f9e25d398 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.400294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee7a4c-84b9-46c9-b0f9-e83f9e25d398 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.401737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 575/1154] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:04.406952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.407341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.407514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.424577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:04.426529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:04.491340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 580/1155] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:04.514759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee7a4c-84b9-46c9-b0f9-e83f9e25d398 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.617850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee7a4c-84b9-46c9-b0f9-e83f9e25d398 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.646382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:04.651094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.651823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.652106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.652383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.662887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:31:04.670330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:31:04.671415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:31:04.672283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 576/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:31:04.678328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.711395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:04.775276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65cd51-6382-44c4-81ea-32f84000359c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.808568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.809231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.809522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.809799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.810542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:04.810850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.812214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.820338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 581/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:04.826335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.929908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daae219-0f7e-43a9-803e-9b5c1d78360f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.963739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.964642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.964930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.965215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] PATH_INFO: `/v3/users/4a4e10c26ca649dbbeb3afc60d06f830` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.966393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] RBAC: Authorizing `identity:get_user(user_id=4a4e10c26ca649dbbeb3afc60d06f830)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.967830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.969487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 577/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/users/4a4e10c26ca649dbbeb3afc60d06f830 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:04.976079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:05.070352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:05.074256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc0b00-f834-425b-92d1-b822cc4cd059 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:05.107552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:05.108193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:05.108477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:05.108785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:05.174132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:05.175732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:05.268306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:05.273280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 582/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:31:05.279325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:05.339236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:05.381864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e48401-4a54-4c8b-9ab9-a90040880e04 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:05.415455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08f6f9-eaec-4306-9596-9de62241705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:05.416309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f6f9-eaec-4306-9596-9de62241705a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:05.416624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f6f9-eaec-4306-9596-9de62241705a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:05.416913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08f6f9-eaec-4306-9596-9de62241705a None admin] PATH_INFO: `/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:05.418100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08f6f9-eaec-4306-9596-9de62241705a None admin] RBAC: Authorizing `identity:get_project(project_id=2919ac5ecfeb449bbbef9f55b85ae5d3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:05.419547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08f6f9-eaec-4306-9596-9de62241705a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:05.421272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 578/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:31:05 2022] GET /identity/v3/projects/2919ac5ecfeb449bbbef9f55b85ae5d3 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:05.428393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:05.525786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed29cd-6114-48ae-bbae-8da232544279 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:05.560199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:05.560886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:05.561180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:05.561465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:05.562291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=2919ac5ecfeb449bbbef9f55b85ae5d3, user.id=4a4e10c26ca649dbbeb3afc60d06f830) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:05.562604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:05.564062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1498dd7-b8a3-4134-b3b2-72415aff3456 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:05.570979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 583/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:31:05 2022] GET /identity/v3/role_assignments?user.id=4a4e10c26ca649dbbeb3afc60d06f830&scope.project.id=2919ac5ecfeb449bbbef9f55b85ae5d3&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:07.898169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8d52-d890-4ead-b38b-eb92c2b17214 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:07.898628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8d52-d890-4ead-b38b-eb92c2b17214 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:07.898740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8d52-d890-4ead-b38b-eb92c2b17214 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:07.900068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 579/1162] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:07.905476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:07.905887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:07.906041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:07.923384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:07.924618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:07.983628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:07.995150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 584/1163] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:08.018554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa8d52-d890-4ead-b38b-eb92c2b17214 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:08.123769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa8d52-d890-4ead-b38b-eb92c2b17214 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:08.152479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:08.161515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:08.162181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:08.162466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:08.162741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:08.232034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:08.233515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:08.338362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 580/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:31:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:31:08.344647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:08.442036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06465f70-868d-4268-aa1e-ee09c0cfa947 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:08.475086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:08.475882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:08.476166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:08.476441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:08.482957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:08.484387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-572da6a3-b416-4001-aa0e-b4532bc517b1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:31:08.499470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:31:08.500711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:31:08.501736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 585/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:31:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:31:08.507065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:08.534917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:08.607147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e680fd-18b9-4497-9440-d0d668c8a09c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:08.639976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:08.640682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:08.640970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:08.641249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:08.641972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:08.642283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:08.644279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cadadf-88c8-401c-9d60-17b8d8656288 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:08.651905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 581/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Fri May 27 03:31:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:10.909997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189526b-c6dd-45df-9fcb-a48dcbf2dc93 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.910521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189526b-c6dd-45df-9fcb-a48dcbf2dc93 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.910619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189526b-c6dd-45df-9fcb-a48dcbf2dc93 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.911988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 586/1167] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:10.917282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66c1d7-440f-424c-8cc7-48f75fd2da6b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.917730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66c1d7-440f-424c-8cc7-48f75fd2da6b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.917834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66c1d7-440f-424c-8cc7-48f75fd2da6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.935798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-aa66c1d7-440f-424c-8cc7-48f75fd2da6b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:10.937056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-aa66c1d7-440f-424c-8cc7-48f75fd2da6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:11.007987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 582/1168] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:11.031448 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189526b-c6dd-45df-9fcb-a48dcbf2dc93 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.085507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:11.133127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189526b-c6dd-45df-9fcb-a48dcbf2dc93 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.166108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c0d65-70c9-44cd-b8bc-6ee6d9e87983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.166758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c0d65-70c9-44cd-b8bc-6ee6d9e87983 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.167042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c0d65-70c9-44cd-b8bc-6ee6d9e87983 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.167321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c0d65-70c9-44cd-b8bc-6ee6d9e87983 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.168068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c0d65-70c9-44cd-b8bc-6ee6d9e87983 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.169462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c0d65-70c9-44cd-b8bc-6ee6d9e87983 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.236475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 587/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Fri May 27 03:31:11 2022] POST /identity/v3/domains => generated 267 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:13.556605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5737117-4b84-4213-9c68-7ebf6f75ce89 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.557191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5737117-4b84-4213-9c68-7ebf6f75ce89 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.557326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5737117-4b84-4213-9c68-7ebf6f75ce89 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.558628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 583/1170] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:13.564056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.564530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.564649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.582688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:13.583937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:13.656428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 588/1171] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:13.681713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5737117-4b84-4213-9c68-7ebf6f75ce89 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.707323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:13.787425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5737117-4b84-4213-9c68-7ebf6f75ce89 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.823047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60b6ff-d061-4d2f-9354-d3cfc7392b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.823818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60b6ff-d061-4d2f-9354-d3cfc7392b0a None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.824102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60b6ff-d061-4d2f-9354-d3cfc7392b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.824377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60b6ff-d061-4d2f-9354-d3cfc7392b0a None admin] PATH_INFO: `/v3/domains/b9ffc28265a548b0af03077eaae07286` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.835750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60b6ff-d061-4d2f-9354-d3cfc7392b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9ffc28265a548b0af03077eaae07286)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.837240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60b6ff-d061-4d2f-9354-d3cfc7392b0a None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.838905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 584/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/domains/b9ffc28265a548b0af03077eaae07286 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:13.845116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.890712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:13.950724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e0bce-d341-47e8-b15f-5aed2f15da85 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.985595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2da06-b963-473a-bb19-ac14c002593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.986246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2da06-b963-473a-bb19-ac14c002593d None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.986530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2da06-b963-473a-bb19-ac14c002593d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.986806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2da06-b963-473a-bb19-ac14c002593d None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.987569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2da06-b963-473a-bb19-ac14c002593d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.989036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2da06-b963-473a-bb19-ac14c002593d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:14.099942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 589/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/projects => generated 361 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:16.334154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dcddc-9c3d-4650-a7bf-118dd0ea56be None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:16.334697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dcddc-9c3d-4650-a7bf-118dd0ea56be None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:16.334697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dcddc-9c3d-4650-a7bf-118dd0ea56be None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:16.336102 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 585/1174] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:16.341845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:16.342362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:16.342458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:16.359642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:16.360835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:16.436892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 590/1175] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:16.461851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dcddc-9c3d-4650-a7bf-118dd0ea56be None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:16.545965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:16.564806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dcddc-9c3d-4650-a7bf-118dd0ea56be None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:16.598409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379500b-7973-4107-86eb-1d8a8379fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:16.599178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379500b-7973-4107-86eb-1d8a8379fba6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:16.599460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379500b-7973-4107-86eb-1d8a8379fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:16.599732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379500b-7973-4107-86eb-1d8a8379fba6 None admin] PATH_INFO: `/v3/domains/b9ffc28265a548b0af03077eaae07286` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:16.600926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379500b-7973-4107-86eb-1d8a8379fba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b9ffc28265a548b0af03077eaae07286)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:16.602354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379500b-7973-4107-86eb-1d8a8379fba6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:16.603956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 586/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Fri May 27 03:31:16 2022] GET /identity/v3/domains/b9ffc28265a548b0af03077eaae07286 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:16.609717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:16.634970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:16.708753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0d166-266c-480b-a268-ab02b05c50f0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:16.743162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceb935-e9ee-4cea-94f7-9a5337e620dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:16.743829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceb935-e9ee-4cea-94f7-9a5337e620dc None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:16.744113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceb935-e9ee-4cea-94f7-9a5337e620dc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:16.744391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceb935-e9ee-4cea-94f7-9a5337e620dc None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:16.745128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceb935-e9ee-4cea-94f7-9a5337e620dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:16.746562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ceb935-e9ee-4cea-94f7-9a5337e620dc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:16.825627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 591/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:31:16 2022] POST /identity/v3/users => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:19.128725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080bb7b-550d-465d-a6fc-e69a7b927d89 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.129281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080bb7b-550d-465d-a6fc-e69a7b927d89 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.129281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080bb7b-550d-465d-a6fc-e69a7b927d89 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.130757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 587/1178] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:19.136150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.136619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.136778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.154046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:19.155244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:19.227515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 592/1179] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:19.251962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080bb7b-550d-465d-a6fc-e69a7b927d89 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.359119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080bb7b-550d-465d-a6fc-e69a7b927d89 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:19.396169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:19.396926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.397211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.397488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.408012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:31:19.417211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:31:19.418469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:31:19.419741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 588/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:31:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:31:19.424528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.436205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:19.525695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe7be3-a8f4-4094-92ef-16280a596465 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:19.560662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:19.561314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.561595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.561869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.562605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:19.562911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:19.564277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:19.572599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 593/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:31:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:19.578511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.591321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:19.677377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6aae8-d914-4f9f-a2f6-02f4d659bcfa None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:19.709500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:19.710362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.710642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.710911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] PATH_INFO: `/v3/users/391db599e5fb4c889a9f73cf5984efcd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.724199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] RBAC: Authorizing `identity:get_user(user_id=391db599e5fb4c889a9f73cf5984efcd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:19.725688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:19.727348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 589/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:31:19 2022] GET /identity/v3/users/391db599e5fb4c889a9f73cf5984efcd => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:19.733339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.806916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:19.836425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806c0da-6528-4cd1-8c88-79564ae9f5a9 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:19.870632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:19.871283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.871573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.871859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.940745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:19.942248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.010502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:20.041843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 594/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:31:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:31:20.047774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:20.114589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:20.148623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3e67f-66fd-4f3f-80a4-6777d7762326 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.181613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5249999-5f7c-486f-8d1d-7c02e0351c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.182466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249999-5f7c-486f-8d1d-7c02e0351c79 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.182753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249999-5f7c-486f-8d1d-7c02e0351c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.183082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5249999-5f7c-486f-8d1d-7c02e0351c79 None admin] PATH_INFO: `/v3/projects/ed94956174b241d0810b128c828a85d9` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.184337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5249999-5f7c-486f-8d1d-7c02e0351c79 None admin] RBAC: Authorizing `identity:get_project(project_id=ed94956174b241d0810b128c828a85d9)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.185812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5249999-5f7c-486f-8d1d-7c02e0351c79 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.187504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 590/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:31:20 2022] GET /identity/v3/projects/ed94956174b241d0810b128c828a85d9 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:20.194503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:20.293666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c0d59-cdc1-407f-b836-6736d53ce50d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.327217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.327862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.328142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.328415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.329263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=ed94956174b241d0810b128c828a85d9, user.id=391db599e5fb4c889a9f73cf5984efcd) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:20.329573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.331667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2ecc7-1308-4186-b877-826356b42abf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.337231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 595/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:31:20 2022] GET /identity/v3/role_assignments?user.id=391db599e5fb4c889a9f73cf5984efcd&scope.project.id=ed94956174b241d0810b128c828a85d9&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:22.594561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1b56-fd92-4a49-bfe6-cb489b04e966 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.595117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1b56-fd92-4a49-bfe6-cb489b04e966 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.595117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1b56-fd92-4a49-bfe6-cb489b04e966 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.596526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 591/1186] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:22.602478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.602909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.603037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.619369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:22.620567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.651018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:22.687673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 596/1187] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:31:22.711683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd1b56-fd92-4a49-bfe6-cb489b04e966 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.816408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd1b56-fd92-4a49-bfe6-cb489b04e966 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.850191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.850895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.851181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.851459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.854158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:22.861983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:31:22.870493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:31:22.871905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:31:22.873126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 592/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:31:22.878938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.978318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb219b-47ed-4d96-ab63-b1d657634acf None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.012553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.013221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.013498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.013775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.014517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:23.014824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.016189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.025244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 597/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:23.031361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.140080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554c0fc-25eb-425d-b7ab-9636cbff5429 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.177116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.178019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.178309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.178586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] PATH_INFO: `/v3/users/391db599e5fb4c889a9f73cf5984efcd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.180225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] RBAC: Authorizing `identity:get_user(user_id=391db599e5fb4c889a9f73cf5984efcd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.181880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.183529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 593/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:31:23 2022] GET /identity/v3/users/391db599e5fb4c889a9f73cf5984efcd => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:23.189536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.228856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:23.326704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fb3a3-a904-47d6-874b-3e5085e422ce None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.372514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e80f33-0f9f-4c5c-a15a-75a8cbb295ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.373396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80f33-0f9f-4c5c-a15a-75a8cbb295ae None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.373692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80f33-0f9f-4c5c-a15a-75a8cbb295ae None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.374222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e80f33-0f9f-4c5c-a15a-75a8cbb295ae None admin] PATH_INFO: `/v3/projects/ed94956174b241d0810b128c828a85d9` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.375418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80f33-0f9f-4c5c-a15a-75a8cbb295ae None admin] RBAC: Authorizing `identity:get_project(project_id=ed94956174b241d0810b128c828a85d9)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.378511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e80f33-0f9f-4c5c-a15a-75a8cbb295ae None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.380380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 598/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:31:23 2022] GET /identity/v3/projects/ed94956174b241d0810b128c828a85d9 => generated 361 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:23.386389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.453507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:23.500299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08de9e-25fd-4266-89fb-d6484bc73563 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.552492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0422f5-066f-459b-bbe0-70e944dcd7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.554493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0422f5-066f-459b-bbe0-70e944dcd7fb None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.554493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0422f5-066f-459b-bbe0-70e944dcd7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.554493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0422f5-066f-459b-bbe0-70e944dcd7fb None admin] PATH_INFO: `/v3/projects/ed94956174b241d0810b128c828a85d9/users/391db599e5fb4c889a9f73cf5984efcd/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.555932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0422f5-066f-459b-bbe0-70e944dcd7fb None admin] RBAC: Authorizing `identity:create_grant(project_id=ed94956174b241d0810b128c828a85d9, user_id=391db599e5fb4c889a9f73cf5984efcd, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.557389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0422f5-066f-459b-bbe0-70e944dcd7fb None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.642398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 594/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:31:23 2022] PUT /identity/v3/projects/ed94956174b241d0810b128c828a85d9/users/391db599e5fb4c889a9f73cf5984efcd/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:25.891600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289ff39-2b8a-441e-82fa-bb4801781e24 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.892086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289ff39-2b8a-441e-82fa-bb4801781e24 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.892175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289ff39-2b8a-441e-82fa-bb4801781e24 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.893817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 599/1193] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:25.899567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.899995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.900124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.917348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:25.918583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:25.986330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 595/1194] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:31:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:31:26.011504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289ff39-2b8a-441e-82fa-bb4801781e24 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.106647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:26.114717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289ff39-2b8a-441e-82fa-bb4801781e24 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.151784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.152537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.152829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.153111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.166388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:31:26.174352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:31:26.175989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:31:26.176877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:31:26.177635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 600/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:31:26.184001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.247458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:26.297832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6020d5-4513-4692-b9de-6e47f8c3c67f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.334251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.334914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.335195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.335469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.336208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:26.336536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.337909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.346199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 596/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:26.352164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.453541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded6d5-80ac-499a-a465-a810856af050 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.488717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.489587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.489874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.490156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] PATH_INFO: `/v3/users/391db599e5fb4c889a9f73cf5984efcd` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.491362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] RBAC: Authorizing `identity:get_user(user_id=391db599e5fb4c889a9f73cf5984efcd)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.492844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.494508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 601/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/users/391db599e5fb4c889a9f73cf5984efcd => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:26.500707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.606170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b0a7c-a0c2-4c92-9beb-59caa7f99aed None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.623918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:26.654083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.654741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.655030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.655317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.721318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.722685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.822988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 597/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:31:26.828926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.854141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:26.929923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799fa1-6f6b-4621-b8dc-1a9e67b330ae None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.963145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4631e-db5f-4d7a-84aa-a574b6c20c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.964002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4631e-db5f-4d7a-84aa-a574b6c20c2d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.964288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4631e-db5f-4d7a-84aa-a574b6c20c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.964594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4631e-db5f-4d7a-84aa-a574b6c20c2d None admin] PATH_INFO: `/v3/projects/ed94956174b241d0810b128c828a85d9` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.965783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4631e-db5f-4d7a-84aa-a574b6c20c2d None admin] RBAC: Authorizing `identity:get_project(project_id=ed94956174b241d0810b128c828a85d9)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.967215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4631e-db5f-4d7a-84aa-a574b6c20c2d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.968950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 602/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/projects/ed94956174b241d0810b128c828a85d9 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:26.975783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.993351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:27.073052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd1798-4664-45d8-ab49-5044652f3961 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:27.105969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:27.106608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:27.106896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:27.107162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:27.107977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=ed94956174b241d0810b128c828a85d9, user.id=391db599e5fb4c889a9f73cf5984efcd) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:27.108283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:27.109742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7d0c0-e365-4900-ba6e-1f94767cd5ea None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:27.116002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 598/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/role_assignments?user.id=391db599e5fb4c889a9f73cf5984efcd&scope.project.id=ed94956174b241d0810b128c828a85d9&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:36.958904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=54 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:37.113387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=60 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:47.382240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aad65-fd9e-47b9-9628-e24773446181 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:47.382822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aad65-fd9e-47b9-9628-e24773446181 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:47.382822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535aad65-fd9e-47b9-9628-e24773446181 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:47.384164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 603/1201] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:32:47.389616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:47.390086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:47.390185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:47.408316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:47.414087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:47.486803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 599/1202] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:32:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:32:47.512834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aad65-fd9e-47b9-9628-e24773446181 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:47.616092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535aad65-fd9e-47b9-9628-e24773446181 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:47.650763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:47.651481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:47.651769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:47.652046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:47.662641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:47.664093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:32:47.682713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:47.683849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:47.685242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:32:47.685242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:32:47.685242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:47.685242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:32:47.685242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 604/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:47.690602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:47.794451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e94f1-8671-4746-8b8f-647483d560bc None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:47.823522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:47.829348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:47.830005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:47.830285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:47.830556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:47.831268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:47.831570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:47.832984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a1cf4-42cf-4c17-b43a-adbab46d08a2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:47.844151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 600/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:47.850316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:47.905911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:47.963269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad37e2-4586-499c-9df1-819293b25b16 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:47.998843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfae9af-e8d9-4a59-8e2c-cb4a89cd4d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:47.999599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfae9af-e8d9-4a59-8e2c-cb4a89cd4d27 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:47.999804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfae9af-e8d9-4a59-8e2c-cb4a89cd4d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:48.000083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfae9af-e8d9-4a59-8e2c-cb4a89cd4d27 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:48.000991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfae9af-e8d9-4a59-8e2c-cb4a89cd4d27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:48.002526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfae9af-e8d9-4a59-8e2c-cb4a89cd4d27 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:48.114426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 605/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Fri May 27 03:32:47 2022] POST /identity/v3/users => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:50.452248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af81c6-8b24-4f19-a426-b859bc400c92 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:50.452841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af81c6-8b24-4f19-a426-b859bc400c92 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:50.452841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af81c6-8b24-4f19-a426-b859bc400c92 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:50.454198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 601/1206] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:32:50.459694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:50.460154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:50.460279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:50.477521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:50.480599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:50.549149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 606/1207] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:32:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:32:50.572094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af81c6-8b24-4f19-a426-b859bc400c92 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:50.675625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af81c6-8b24-4f19-a426-b859bc400c92 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:50.709160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:50.709879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:50.710165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:50.710441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:50.721825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:32:50.729546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:50.733800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:50.734876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 602/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:32:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:50.737334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:50.776416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:50.834597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499cd75-69f2-46aa-bed8-36114311f623 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:50.868173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:50.868913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:50.869238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:50.869515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:50.870260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:50.870565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:50.871941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:50.881961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 607/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:50.887666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:50.940358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:50.990325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4401243-7cf2-40fd-a11c-63ea0777199b None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:51.024403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4988-c959-430f-a256-55411939010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:51.029817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4988-c959-430f-a256-55411939010e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.029817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4988-c959-430f-a256-55411939010e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.029817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4988-c959-430f-a256-55411939010e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:51.036459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db4988-c959-430f-a256-55411939010e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:51.037955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db4988-c959-430f-a256-55411939010e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-c2db4988-c959-430f-a256-55411939010e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:32:51.054566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:51.055478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:51.056397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:32:51.056397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:32:51.056397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:51.056397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:51.056397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 603/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:51.062278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:51.163284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:51.166332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b725d46-9873-4c06-8a62-4bbf9dd82d7a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:51.200194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:51.200888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.201215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.201489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:51.202205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:51.202513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:51.204580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:51.215888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 608/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:51.221517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4988-c959-430f-a256-55411939010e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:51.322210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4988-c959-430f-a256-55411939010e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:51.323645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:51.356050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:51.356897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.357179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.357456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:51.365097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:51.366536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:32:51.382079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:51.383262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:51.384461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 604/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:32:51.389082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:51.488999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0279-7355-46a5-ae6a-83ebf1abba59 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:51.523105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:51.523736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.524098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.524438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:51.525356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:51.525665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:51.527122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:51.544870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 609/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Fri May 27 03:32:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:51.551647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:51.650177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93265-3895-4729-a0bc-ad7b2822762c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:51.684177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:51.689070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:51.689070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:51.689070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:51.715588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:51.754840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:51.756334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:51.860938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 605/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:32:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:32:51.866995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:51.887772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:51.971478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfd9be-d47f-40c6-a5cd-ff975d6df818 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:52.006083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:52.006889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.007166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.007453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.018518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.020033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:32:52.038147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:52.039251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:52.040433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:32:52.040433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:32:52.040433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:52.040433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:32:52.040433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 610/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:52.044999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:52.107052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:52.146510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be628bce-b575-467a-bf74-bf41a2098604 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:52.181303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:52.181975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.182264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.182545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.183272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:52.183582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.185051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.197365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 606/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:52.203082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:52.274949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:52.305362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0402a9e-b594-402c-a464-7803a5e85c5b None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:52.338436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:52.339224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.339513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.339790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.347010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.349029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:32:52.363167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:52.364233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:32:52.365189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 611/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:52.370422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:52.470680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141475f-41c0-4f12-a863-25e33661e31f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:52.493575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:52.506011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:52.506684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.506969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.507248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.508071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:52.508383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.509874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5dcee-6fcc-429c-afed-69a9f1996332 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.523056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 607/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:52.535363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:52.634287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2e397-116d-40df-803a-0a4d48bf616d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:52.662833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:52.668324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:52.668999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:52.669280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:52.669557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:52.670378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=4f611173abbd4a89b4f762571a760a27) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:52.670684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:52.672254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8746c-3bc0-44be-b416-c736edc1c83e None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:52.677840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 612/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:32:52 2022] GET /identity/v3/role_assignments?user.id=4f611173abbd4a89b4f762571a760a27&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:54.940154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e623-d93a-48cf-8e3c-bfd45c9387c6 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:54.940717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e623-d93a-48cf-8e3c-bfd45c9387c6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:54.940717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190e623-d93a-48cf-8e3c-bfd45c9387c6 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:54.942125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 608/1220] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:32:54.947920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:54.948477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:54.948477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:54.966043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:54.967228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:55.035503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 613/1221] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:32:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:32:55.058595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190e623-d93a-48cf-8e3c-bfd45c9387c6 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:55.167158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190e623-d93a-48cf-8e3c-bfd45c9387c6 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:55.204238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:55.205078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:55.205367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:55.205650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:55.216916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:32:55.224694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:55.225922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:55.226798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 609/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:55.232060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:55.333375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e3e93-d608-4c10-94a4-f037670a8135 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:55.356460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:55.367776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:55.372952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:55.372952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:55.372952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:55.372952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:55.372952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:55.372952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:55.381071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 614/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:55.386993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:55.435483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:55.495874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a303c3-1db6-4ae2-9ecf-3415277b092c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:55.536481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:55.537302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:55.537582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:55.537855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:55.550511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:55.553694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:32:55.571191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:55.572444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:55.573531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:32:55.573531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:32:55.573531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:55.573531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:55.573531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 610/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:55.578456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:55.682440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f4d69-48b9-435c-975a-d62c84b9b7bc None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:55.717406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:55.717993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:55.718283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:55.718569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:55.719799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:55.720102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:55.721857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:55.733909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 615/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:55.740191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:55.838281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:55.845884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54796f5f-08d5-44d4-bb1d-25787eca23ca None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:55.883389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:55.884323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:55.884639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:55.884936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:55.892744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:55.894226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:32:55.910043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:55.912927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:55.914093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 611/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:55.917505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:55.964925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:56.018879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56eef4-d929-4080-8809-0cc66d8388d3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:56.055685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:56.056363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:56.056683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:56.056963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:56.057920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:56.058310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:56.059875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:56.078890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 616/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Fri May 27 03:32:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:56.085118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:56.194349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73a857-5ad2-4b42-af04-0bb9f28868ae None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:56.230039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:56.230838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:56.231120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:56.231395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:56.242767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:56.244229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:32:56.262101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:56.263148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:56.264201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:32:56.264201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:32:56.264201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:56.264201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:56.264201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 612/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:56.273034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:56.372991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20b6e9-aed5-4857-ba2d-d26d5706b22d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:56.375589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:56.406222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:56.407381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:56.407666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:56.407941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:56.408699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:56.409010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:56.410412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:56.422067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 617/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:56.429206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:56.438204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:56.534536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2fa44-744a-4eca-ab41-e9c456ce2319 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:56.568840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:56.569632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:56.569914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:56.570190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:56.576914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:56.578360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:32:56.592797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:56.593871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:32:56.595919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 613/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:56.600478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:56.704771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee07b7-110d-4f3d-b01d-22c65e9856df None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:56.739282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:56.739959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:56.740244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:56.740552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:56.741361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:56.741672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:56.743141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c19b3-8a72-49a2-aaf6-00dcc7721935 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:56.757311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 618/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:32:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:32:56.763343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:56.831771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:56.867066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0c0bc-aaf6-46aa-b8f8-c4d5316281d4 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:56.902944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8067-9284-428e-ba7a-61e002092285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:56.903836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8067-9284-428e-ba7a-61e002092285 None admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:56.904119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8067-9284-428e-ba7a-61e002092285 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:56.904395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8067-9284-428e-ba7a-61e002092285 None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/4f611173abbd4a89b4f762571a760a27/roles/f7cba27be2b3434687d97625de185fbc` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:56.919578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8067-9284-428e-ba7a-61e002092285 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=4f611173abbd4a89b4f762571a760a27, role_id=f7cba27be2b3434687d97625de185fbc)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:56.921094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8067-9284-428e-ba7a-61e002092285 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:57.013793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 614/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:32:56 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/4f611173abbd4a89b4f762571a760a27/roles/f7cba27be2b3434687d97625de185fbc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:59.286509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6913ab-d805-4324-9d98-bebc014874cc None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:59.287121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6913ab-d805-4324-9d98-bebc014874cc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:59.287121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6913ab-d805-4324-9d98-bebc014874cc None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:59.288492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 619/1233] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:32:59.293978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:59.294384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:59.294580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:59.311889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:59.313732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:59.382463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 615/1234] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:32:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:32:59.407418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6913ab-d805-4324-9d98-bebc014874cc None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:59.432097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:59.510463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6913ab-d805-4324-9d98-bebc014874cc None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:59.546265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:59.546994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:59.547767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:59.548048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:59.559195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:32:59.567683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:59.570786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:32:59.571685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 620/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Fri May 27 03:32:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:59.574946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:59.621824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:59.674678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8116f-168e-4a60-af60-da6c77064006 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:59.709390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:59.710062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:59.710351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:59.710633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:59.711384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] RBAC: Adding query filter params (name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:59.711696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:59.713107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:59.721303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 616/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Fri May 27 03:32:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:32:59.727127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:59.816737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:59.832465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e41b2-1abe-4525-a1b0-3f8fec9b154d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:59.866434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:59.867216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:59.867541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:59.867819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:59.879651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:59.881171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:32:59.900156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:32:59.901322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:32:59.902353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:32:59.902353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:32:59.902353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:32:59.902353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:32:59.902353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 621/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:32:59.907884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:00.010173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02aa574-c339-4466-9840-a3ce64e56381 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:00.019911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:00.044873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:00.045542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:00.045825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:00.046105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:00.046852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:00.047160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:00.049130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:00.060791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 617/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:00.066518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:00.171994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe710518-0a87-40c9-8c18-e1d0b85d07bf None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:00.207326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:00.208170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:00.208458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:00.208756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:00.211436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:00.216044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:00.217551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:33:00.232635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:00.233825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:00.235772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 622/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:00.240605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:00.345537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c1ecd-0faf-41ce-8282-267a803a7ac6 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:00.381173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:00.381879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:00.382167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:00.382476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:00.383346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:00.383657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:00.385238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:00.403549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 618/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Fri May 27 03:33:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:00.410221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:00.510889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968be25c-4e17-4a66-bf44-2921be56d81d None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:00.545675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:00.546325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:00.546610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:00.546886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:00.610476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:00.617571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:00.619031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:00.721453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 623/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:33:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:33:00.727315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:00.755733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:00.842720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed19b3-0011-4c6a-ad37-816d59aebd03 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:00.880049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:00.880882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:00.881170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:00.881437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:00.893195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:00.894666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:00.913120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:00.914182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:00.915101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:00.915101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:00.915101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:00.915101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:00.915101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 619/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:00.920431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:01.016655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:01.022456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdadcf-cac2-4d50-8896-10a6a042d09a None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:01.058167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:01.058830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:01.059118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:01.059392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:01.060116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:01.060421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:01.061824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:01.073923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 624/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:01.080326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:01.169711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:01.191963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751ddcc-d11b-469d-9483-a9a42b7f72d0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:01.226304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:01.227090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:01.227367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:01.227635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:01.234446 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:01.235940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:33:01.249877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:01.251141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:01.252374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 620/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:01.257062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:01.367692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf300bc9-74cd-4ec8-933a-9b88d0166dad None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:01.401748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:01.402450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:01.402739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:01.403014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:01.403792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:01.404099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:01.405572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484474d-0cfb-4472-a7da-8fe1d341689f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:01.418070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 625/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:01.425585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:01.528215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8372f6-737d-4bd3-bdf1-306efb4058de None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:01.569210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:01.569864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:01.570143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:01.570415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:01.571241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] RBAC: Adding query filter params (role.id=f7cba27be2b3434687d97625de185fbc, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=4f611173abbd4a89b4f762571a760a27) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:01.571551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:01.573012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5477b7-d190-4c71-8130-59e84a8991bf None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:01.579737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 621/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:33:01 2022] GET /identity/v3/role_assignments?user.id=4f611173abbd4a89b4f762571a760a27&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=f7cba27be2b3434687d97625de185fbc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:33:03.901394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10898990-c87f-496b-bbe8-33402034b454 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:03.901901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10898990-c87f-496b-bbe8-33402034b454 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:03.901995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10898990-c87f-496b-bbe8-33402034b454 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:03.903308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 626/1247] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:03.909669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:03.910127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:03.910235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:03.915210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:03.928671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:03.929876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:03.998521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 622/1248] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:04.023527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10898990-c87f-496b-bbe8-33402034b454 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:04.043172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:04.127339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10898990-c87f-496b-bbe8-33402034b454 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:04.167018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:04.167826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:04.168107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:04.168387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:04.178679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:33:04.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:04.189713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:04.190648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 627/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:33:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:04.193730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:04.293746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7e5b6-3b05-43ff-b261-544044f8c2ee None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:04.323902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:04.329433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:04.330105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:04.330393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:04.330672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:04.331420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:04.331727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:04.333139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:04.341254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 623/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:33:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:04.347951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:04.428790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:04.452872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e0f1b-f09b-4756-9e7c-8e59e307dfd4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:04.486540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:04.487334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:04.487617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:04.487892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:04.498820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:04.503955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:04.519649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:04.520770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:04.535049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:04.535049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:04.535049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:04.535049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:04.535049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 628/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:04.535406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:04.626713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ed11-77f4-4887-8555-575608e769a8 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:04.661537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:04.662193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:04.662474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:04.662747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:04.663457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:04.663762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:04.665219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:04.676874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 624/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:33:04.683690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:04.783196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdad587-66b9-47f4-a996-b4ca4d0e97b2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:04.809875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:04.818103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:04.818903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:04.819187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:04.819462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:04.826983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:04.828450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:33:04.842714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:04.844014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:04.845396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 629/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:04.849839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:04.910331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:04.956709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908382a-b5df-4906-bf3b-b4440b1ac1a0 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:04.991430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:04.992104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:04.992390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:04.992697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:04.993533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:04.993846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:04.995288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.014438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 625/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Fri May 27 03:33:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:05.020489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:05.122950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcca61-353e-46e7-a122-82ca1c67e0ce None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:05.158460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:05.159106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.159388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.159663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.215366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:05.228296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:05.229815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.325234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 630/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:33:05.331837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:05.432207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0081fa9-5d3a-44f6-9c02-8138dd814e22 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:05.445683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:05.466697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:05.467411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.467692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.467965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.478768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:05.480412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:05.499211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:05.500234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:05.501384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:05.501384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:05.501384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:05.501384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:05.501384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 626/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:05.509527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:05.601665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:05.613649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d06-2d4b-4bf8-99af-66f0478935f0 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:05.648153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:05.648846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.649135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.649417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.650148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:05.650456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:05.651987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.664082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 631/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:05.669958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:05.775711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8d70d-2e7c-4cb8-a5a1-200a064cad72 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:05.810581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:05.811346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.811627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.811894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.818951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:05.820451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:33:05.835328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:05.842627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:05.843549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 627/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:05.844730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:05.944419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ebebb-8a64-4975-af0d-e88852da8655 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:05.977570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:05.978221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:05.979110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:05.979110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:05.979600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:05.979906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:05.981367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9722c4d-3da0-4e0d-845b-c8f96111aa77 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:05.993407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 632/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:06.000084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:06.029234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:06.106921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017c23-99df-4f07-ba76-b7b943193436 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:06.140462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:06.141750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:06.142035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:06.142315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:06.143152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=4f611173abbd4a89b4f762571a760a27) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:06.143469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:06.144961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d84f70-24b1-4e49-84b3-a0fdf239c72e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:06.150546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 628/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:33:05 2022] GET /identity/v3/role_assignments?user.id=4f611173abbd4a89b4f762571a760a27&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:08.331874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d931b2-4728-42d1-a08e-13af42ff3f91 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.332427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d931b2-4728-42d1-a08e-13af42ff3f91 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.332427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d931b2-4728-42d1-a08e-13af42ff3f91 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.333871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 633/1261] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:33:08.339389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.339952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.340042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.357724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:08.358901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:08.427539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 629/1262] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:08.450432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d931b2-4728-42d1-a08e-13af42ff3f91 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:08.461511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:08.558016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d931b2-4728-42d1-a08e-13af42ff3f91 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:08.595098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:08.595823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.596112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.596396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.609214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:33:08.617619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:08.619817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:08.620955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 634/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:33:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:08.625741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:08.644956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:08.724747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8680c31-6d76-4f65-9884-802d76893171 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:08.759144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:08.759838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.760184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.760467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.761298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:08.761636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:08.763046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:08.773038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 630/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:33:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:08.779117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:08.849199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:08.886719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aefbc-0769-4cd8-a51e-e1192f30e9c2 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:08.920386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:08.921208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.921493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.921772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.933139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:08.934607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:08.952447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:08.957966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:08.959086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:08.959086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:08.959086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:08.959086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:08.959086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 635/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:08.961429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.044389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:09.064747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875b9c-3a5f-4998-b982-73b8bdc09c4e None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.099889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.100481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.100802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.101083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.101807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:09.102113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.103487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.115567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 631/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:09.121902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.233008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4c90-687d-49a6-bfc1-2b3747dcff10 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.254015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:09.268553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.269367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.269663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.269960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.278083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.279542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:33:09.294113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:09.296106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:09.298818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 636/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:09.303157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.408292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa835b-a36a-4c20-9f37-26c4a9f38d62 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.443680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.444352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.444670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.444951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.445791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:09.446108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.447566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.458044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:09.466153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 632/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:09.472227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.575162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c98180-bb5e-4766-bfe6-bf39b48e5ee3 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.608986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.610394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.610820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.611100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.621567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.623297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:09.642329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:09.643572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:09.644631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:09.644631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:09.644631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:09.644631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:09.644631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 637/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:09.649773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.750337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f44e-c8a4-448e-986d-643896a0501f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.785562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.786226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.786509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.786784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.787495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:09.787798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.789230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.801374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 633/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:09.807411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.838446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:09.907671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a860e1-4029-4280-9a32-e95649fe9076 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.942911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da79d-8233-488f-8146-6841864637df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.943734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da79d-8233-488f-8146-6841864637df None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.944014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da79d-8233-488f-8146-6841864637df None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.944288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1da79d-8233-488f-8146-6841864637df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.950816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da79d-8233-488f-8146-6841864637df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.952268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1da79d-8233-488f-8146-6841864637df None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-6a1da79d-8233-488f-8146-6841864637df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:33:09.966044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:09.967153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:09.968391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 638/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:09.973698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:10.038269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:10.080612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e6f40-aed3-4725-83fc-3fde76bb6895 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:10.113861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:10.114520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:10.114811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:10.115084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:10.115868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:10.116178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:10.117706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff8d86-4c31-462c-9e36-9954e4db7085 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:10.132629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 634/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:10.138811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da79d-8233-488f-8146-6841864637df None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:10.230000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:10.238911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1da79d-8233-488f-8146-6841864637df None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:10.273738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a1761-4a82-4412-b45b-6e95ff3e723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:10.274637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a1761-4a82-4412-b45b-6e95ff3e723f None admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:10.274924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a1761-4a82-4412-b45b-6e95ff3e723f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:10.275205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a1761-4a82-4412-b45b-6e95ff3e723f None admin] PATH_INFO: `/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/4f611173abbd4a89b4f762571a760a27/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:10.277189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a1761-4a82-4412-b45b-6e95ff3e723f None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c9c8b9edfd489d9c51c633882a4cc7, user_id=4f611173abbd4a89b4f762571a760a27, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:10.278641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a1761-4a82-4412-b45b-6e95ff3e723f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:10.364585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 639/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Fri May 27 03:33:10 2022] PUT /identity/v3/projects/c2c9c8b9edfd489d9c51c633882a4cc7/users/4f611173abbd4a89b4f762571a760a27/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:33:12.570452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d44bde-ede0-426f-97e2-6fe0b94e8dae None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:12.571667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d44bde-ede0-426f-97e2-6fe0b94e8dae None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:12.571667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d44bde-ede0-426f-97e2-6fe0b94e8dae None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:12.572561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 635/1274] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:12.577964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:12.579061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:12.579061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:12.595599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:12.596817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:12.662952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 640/1275] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:12.686477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d44bde-ede0-426f-97e2-6fe0b94e8dae None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:12.791238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d44bde-ede0-426f-97e2-6fe0b94e8dae None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:12.809215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:12.828425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:12.828425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:12.828425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:12.828425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:12.838229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:33:12.846630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:12.847702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:12.848649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 636/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Fri May 27 03:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:12.854051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:12.939375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:12.957633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938455a-e512-4aab-a9f6-8453e2577367 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:12.991816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:12.992787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:12.993073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:12.993348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:12.994091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:12.994403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:12.995813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:13.004676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 641/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Fri May 27 03:33:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:13.010345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:13.110800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502119-1247-4f93-9268-442fe43443bb None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:13.144860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:13.145658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:13.145943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:13.146218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:13.157349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:13.158805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:13.176383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:13.177515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:13.178525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:13.178525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:13.178525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:13.178525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:13.178525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 637/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:13.183350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:13.282828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51fa55-67cf-4a51-8545-f8bb2979c1ed None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:13.315801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:13.316449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:13.316764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:13.317037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:13.317760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:13.318050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:13.319399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:13.330970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 642/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:13.336651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:13.368153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:13.435881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8dc83-4a57-4b6f-97db-ab23e14ddda3 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:13.471673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:13.472569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:13.475829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:13.475829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:13.480648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:13.482164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:33:13.498439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:13.499669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:13.501409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 638/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:13.506018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:13.510665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:13.612662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755900f2-2a69-405b-995b-4daddabcb760 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:13.649966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:13.650570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:13.650859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:13.651139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:13.652007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:13.652316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:13.653925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:13.674252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 643/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Fri May 27 03:33:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:33:13.680283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:13.773122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:13.783484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed795c3-8401-40ae-ba5f-733d01b2ac40 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:13.817675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:13.818326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:13.818610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:13.818883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:13.896142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:13.897730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:13.988224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:13.998716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 639/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Fri May 27 03:33:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:33:14.005935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:14.068268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:14.107491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01424ecf-c279-47a2-b7c6-3bfe2ce4b6dd None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:14.142845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:14.143630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:14.143917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:14.144196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:14.156007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:14.157503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:33:14.175934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:14.177133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:14.178253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:33:14.178253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:33:14.178253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:33:14.178253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:14.178253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 644/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:14.182973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:14.285228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969a141-ecd8-40f5-89d6-160904445e4c None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:14.320130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:14.320828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:14.326824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:14.326824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:14.326824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:14.326824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:14.326824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:14.334944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 640/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Fri May 27 03:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:14.340649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:14.440193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca523-5c3b-438c-a498-480fdc9207b4 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:14.454318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:14.473934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:14.474732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:14.475014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:14.475292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:14.481927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:14.485092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:33:14.498403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application with Lock( May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:14.499543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: ERROR keystone.server.flask.application May 27 03:33:14.501171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 645/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Fri May 27 03:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:14.506484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:14.557866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:14.606505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4ac9c-146c-4195-8bea-a75352e9069f None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:14.640685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:14.641733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:14.642046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:14.642316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:14.643105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:14.643420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:14.644903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29684b6d-8296-4c2b-8c48-917a5f7a7082 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:14.658137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 641/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Fri May 27 03:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:33:14.664477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:14.767333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2085c7b-3195-40a7-98be-9d68b3d0c6bb None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:14.801362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:14.802006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:14.802292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:14.802568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:14.803489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] RBAC: Adding query filter params (role.id=7842553ae25f43ec83b0bb89e9fc9175, scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7, user.id=4f611173abbd4a89b4f762571a760a27) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:14.803800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:14.805265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb42d-13af-473a-beb9-74c85b35b46c None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:14.811111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 646/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Fri May 27 03:33:14 2022] GET /identity/v3/role_assignments?user.id=4f611173abbd4a89b4f762571a760a27&scope.project.id=c2c9c8b9edfd489d9c51c633882a4cc7&role.id=7842553ae25f43ec83b0bb89e9fc9175 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:17.075080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6521b4-c3f3-4726-bee7-62b439ca2e39 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:17.075663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6521b4-c3f3-4726-bee7-62b439ca2e39 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:17.075663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6521b4-c3f3-4726-bee7-62b439ca2e39 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:17.077058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 642/1288] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:17.083413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:17.083875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:17.083971 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:17.100484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:17.103789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:17.132806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:17.172456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 647/1289] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:17.196163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6521b4-c3f3-4726-bee7-62b439ca2e39 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:17.300127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6521b4-c3f3-4726-bee7-62b439ca2e39 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:17.310871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:17.334615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:17.335452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:17.335742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:17.336018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:17.336710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:17.338125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:17.350803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:17.352735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:17.354470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 643/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:33:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:17.358181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:17.458213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a042c-5df2-425f-bf52-e1480fe53ec4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:17.493062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:17.493754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:17.494041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:17.494320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:17.495030 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:17.495341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:17.496993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e6aeb-e3b2-40b9-9347-b4a62feadc68 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:17.502253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 648/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Fri May 27 03:33:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:17.508078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:17.608120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504555c6-fd0e-4e08-be7f-95f697308e70 None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:17.644352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:17.645047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:17.645329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:17.645602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] PATH_INFO: `/v3/services` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:17.646393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:17.646699 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:17.648071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af5105-c0dd-49ba-9489-a326cf1e33e5 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:17.652819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 644/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Fri May 27 03:33:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:19.955118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7f9d-b5a3-42a1-ae77-396c34cdd1c1 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:19.955641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7f9d-b5a3-42a1-ae77-396c34cdd1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:19.955641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7f9d-b5a3-42a1-ae77-396c34cdd1c1 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:19.957097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 649/1293] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:19.962725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d1521-31f4-4ccc-8ec2-4d35b3bfeb4c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:19.965758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d1521-31f4-4ccc-8ec2-4d35b3bfeb4c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:19.965758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d1521-31f4-4ccc-8ec2-4d35b3bfeb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:19.980616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-8e6d1521-31f4-4ccc-8ec2-4d35b3bfeb4c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:19.981802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8e6d1521-31f4-4ccc-8ec2-4d35b3bfeb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:20.011623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:20.048562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 645/1294] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:20.072084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7f9d-b5a3-42a1-ae77-396c34cdd1c1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:20.104062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:20.182054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7f9d-b5a3-42a1-ae77-396c34cdd1c1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:20.217664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f43550-4bad-4a21-bdb6-9b9ee270b564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:20.218368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43550-4bad-4a21-bdb6-9b9ee270b564 None admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:20.218658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43550-4bad-4a21-bdb6-9b9ee270b564 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:20.219562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43550-4bad-4a21-bdb6-9b9ee270b564 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:20.220333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f43550-4bad-4a21-bdb6-9b9ee270b564 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:20.221785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f43550-4bad-4a21-bdb6-9b9ee270b564 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:20.312969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 650/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Fri May 27 03:33:20 2022] POST /identity/v3/services => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:33:22.566678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af868fd0-5bc0-496b-bb90-e7c596662a1b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:22.567255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af868fd0-5bc0-496b-bb90-e7c596662a1b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:22.567255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af868fd0-5bc0-496b-bb90-e7c596662a1b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:22.568696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 646/1296] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:22.574061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:22.574481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:22.574632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:22.592087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:22.593374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:22.668589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 651/1297] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:22.691890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af868fd0-5bc0-496b-bb90-e7c596662a1b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:22.798987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af868fd0-5bc0-496b-bb90-e7c596662a1b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:22.834611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:22.835319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:22.835605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:22.835882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:22.836569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:22.837953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:22.850727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:22.852252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:22.853947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 647/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:33:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:22.859900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:22.879867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:22.960634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77055b-e46f-4c45-ac6f-bdd2989cb004 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:22.994747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:22.995509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:22.995805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:22.996088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:22.997078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:22.997388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:22.998762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:23.004829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 652/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Fri May 27 03:33:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:23.011142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:23.043563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:23.113004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ced4a-284d-4335-9804-bcd5c27b367d None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:23.149627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:23.150291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:23.150579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:23.150854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:23.151644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] RBAC: Adding query filter params (interface=public, service_id=c8e268a4eff14d13874e697455e9f611, region_id=RegionOne) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:23.151957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:23.153398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee00f6-9b1c-4ab8-aee5-899e4b94cd51 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:23.161582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 648/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Fri May 27 03:33:23 2022] GET /identity/v3/endpoints?service_id=c8e268a4eff14d13874e697455e9f611&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:23.164781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:23.254295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:23.274284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a0da9-7974-469e-b311-c53215fa6cd8 None admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:23.310358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1079b1b-f629-4fc9-8c9a-02fd717987e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:23.311026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1079b1b-f629-4fc9-8c9a-02fd717987e0 None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:23.311316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1079b1b-f629-4fc9-8c9a-02fd717987e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:23.311598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1079b1b-f629-4fc9-8c9a-02fd717987e0 None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:23.313217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1079b1b-f629-4fc9-8c9a-02fd717987e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:23.314659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1079b1b-f629-4fc9-8c9a-02fd717987e0 None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:23.321844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 653/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Fri May 27 03:33:23 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:33:25.584556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706d0a4-25ad-493c-a789-d6a560061568 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:25.585120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706d0a4-25ad-493c-a789-d6a560061568 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:25.585120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706d0a4-25ad-493c-a789-d6a560061568 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:25.586496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 649/1302] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:25.592728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:25.593217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:25.593722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:25.617060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:25.618449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:25.710841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 654/1303] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:33:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:33:25.739139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706d0a4-25ad-493c-a789-d6a560061568 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:25.860230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:25.867185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706d0a4-25ad-493c-a789-d6a560061568 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:25.908201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:25.909044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:25.909328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:25.909601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:25.910262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:25.911631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:33:25.924381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:33:25.925656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application with Lock( May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self._enter() May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application return self.creator() May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application created_value = creator( May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: ERROR keystone.server.flask.application May 27 03:33:25.926991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 650/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Fri May 27 03:33:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:33:25.931876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:26.017143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:26.040112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4d586-97ab-4ab0-8cc9-56561e72a3ce None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:26.075569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:26.076334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:26.076645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:26.076923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] PATH_INFO: `/v3/services` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:26.077691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:33:26.078042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:26.079409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb2b6b-0ccc-4237-b788-a28cf836452f None admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:26.084802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 655/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Fri May 27 03:33:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:26.090676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:26.193232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb6eb3-cf91-4be2-b084-0c856b2cc3bf None admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:26.230641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cca98-6334-4831-8cb0-82322ff12bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:26.231405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cca98-6334-4831-8cb0-82322ff12bc2 None admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:26.231713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cca98-6334-4831-8cb0-82322ff12bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:26.232046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cca98-6334-4831-8cb0-82322ff12bc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:26.232939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cca98-6334-4831-8cb0-82322ff12bc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:26.234729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cca98-6334-4831-8cb0-82322ff12bc2 None admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:26.345856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 651/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Fri May 27 03:33:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:33:36.081570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=30 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:36.340338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:09.745199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5ffe2-7277-4161-b7f6-7d2a77d178b6 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:09.746275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5ffe2-7277-4161-b7f6-7d2a77d178b6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:09.746779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5ffe2-7277-4161-b7f6-7d2a77d178b6 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:09.754225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e264c7-6fd2-4b91-ae33-4e709099c452 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:09.755220 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e264c7-6fd2-4b91-ae33-4e709099c452 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:09.756343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e264c7-6fd2-4b91-ae33-4e709099c452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:09.760442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 656/1307] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:09 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:09.844105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b4e264c7-6fd2-4b91-ae33-4e709099c452 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:09.845465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b4e264c7-6fd2-4b91-ae33-4e709099c452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:09.988387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 652/1308] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:40:10.065113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1045-9e66-405a-8205-3ad16f2c6854 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:10.065788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1045-9e66-405a-8205-3ad16f2c6854 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:10.066327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1045-9e66-405a-8205-3ad16f2c6854 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:10.069164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 657/1309] 10.222.0.58 () {54 vars in 964 bytes} [Fri May 27 03:40:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:10.075562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427439f5-fb4d-4863-88d1-492263dc8436 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:10.075899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427439f5-fb4d-4863-88d1-492263dc8436 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:10.076128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427439f5-fb4d-4863-88d1-492263dc8436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:10.174898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-427439f5-fb4d-4863-88d1-492263dc8436 None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:10.176165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-427439f5-fb4d-4863-88d1-492263dc8436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:10.202085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:10.275508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 653/1310] 10.222.0.58 () {58 vars in 1061 bytes} [Fri May 27 03:40:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:10.285939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1045-9e66-405a-8205-3ad16f2c6854 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:10.288883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0ded1045-9e66-405a-8205-3ad16f2c6854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:10.561758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1045-9e66-405a-8205-3ad16f2c6854 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:10.659963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:10.660785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:10.661184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:10.661573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:10.663706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:10.741683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:10.813744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:10.815142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b342a-641a-4ee4-9144-bebe7d03b114 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:10.849477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 658/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:13.862583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8d30-db7d-40b2-bb45-7344b0c9398a None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:13.863110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8d30-db7d-40b2-bb45-7344b0c9398a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:13.863110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8d30-db7d-40b2-bb45-7344b0c9398a None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:13.864791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 654/1312] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:13.870996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea33d5-c0ad-4874-8c85-4ad9bea009bb None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:13.871640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea33d5-c0ad-4874-8c85-4ad9bea009bb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:13.871640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea33d5-c0ad-4874-8c85-4ad9bea009bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:13.981514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ffea33d5-c0ad-4874-8c85-4ad9bea009bb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:13.983036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ffea33d5-c0ad-4874-8c85-4ad9bea009bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:14.134327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 659/1313] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:14.188794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8d30-db7d-40b2-bb45-7344b0c9398a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:14.191208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2b7f8d30-db7d-40b2-bb45-7344b0c9398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:14.321324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8d30-db7d-40b2-bb45-7344b0c9398a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:14.441042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:14.445429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:14.446231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:14.446624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:14.447007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:14.449018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:14.613693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:14.615711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d36a5-31dd-485c-95b6-024abe5aee9d service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:14.639703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 655/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:17.534123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace502c-f3a0-46bb-af41-d7db4a960156 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:17.534659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace502c-f3a0-46bb-af41-d7db4a960156 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:17.534659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace502c-f3a0-46bb-af41-d7db4a960156 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:17.536092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 660/1315] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:17.542365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f608e-59f9-4a69-b70e-23049fd4afe2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:17.542844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f608e-59f9-4a69-b70e-23049fd4afe2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:17.542934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f608e-59f9-4a69-b70e-23049fd4afe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:17.610596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5d0f608e-59f9-4a69-b70e-23049fd4afe2 None None] MFA Rules not processed for user `386db4044f5c4138b6858880d5c6bc71`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:17.611823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5d0f608e-59f9-4a69-b70e-23049fd4afe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:17.667395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:17.713474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 656/1316] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:17.766120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b49a28-9ff4-4715-b9ba-90ddcae99dec None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:17.766442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b49a28-9ff4-4715-b9ba-90ddcae99dec None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:17.766723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b49a28-9ff4-4715-b9ba-90ddcae99dec None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:17.768111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 661/1317] 10.222.0.58 () {54 vars in 964 bytes} [Fri May 27 03:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:17.774322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da18d-6c15-4c6f-8e66-f904923186b5 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:17.774614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da18d-6c15-4c6f-8e66-f904923186b5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:17.774894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1da18d-6c15-4c6f-8e66-f904923186b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:17.851424 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7d1da18d-6c15-4c6f-8e66-f904923186b5 None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:17.852633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7d1da18d-6c15-4c6f-8e66-f904923186b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:17.932921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 657/1318] 10.222.0.58 () {58 vars in 1061 bytes} [Fri May 27 03:40:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:40:17.946589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b49a28-9ff4-4715-b9ba-90ddcae99dec None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:17.948139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-73b49a28-9ff4-4715-b9ba-90ddcae99dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:17.957791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:18.063799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b49a28-9ff4-4715-b9ba-90ddcae99dec None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:18.149479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:18.150045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:18.150327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:18.150601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:18.152320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:18.218359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:18.265632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:18.267068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd6b55-8dec-466d-ae2f-eb67cc3d9ea3 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:18.285269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 662/1319] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:40:22.001354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b36b0-6796-42ad-a7f6-fb2268433aa7 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:22.002004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b36b0-6796-42ad-a7f6-fb2268433aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:22.002004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b36b0-6796-42ad-a7f6-fb2268433aa7 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:22.003315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 658/1320] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:22.010683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5757c11-1686-41db-9b95-408472255254 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:22.011213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5757c11-1686-41db-9b95-408472255254 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:22.011213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5757c11-1686-41db-9b95-408472255254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:22.083935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c5757c11-1686-41db-9b95-408472255254 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:22.085247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c5757c11-1686-41db-9b95-408472255254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:22.198377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 663/1321] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:22.238602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b36b0-6796-42ad-a7f6-fb2268433aa7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:22.241181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c07b36b0-6796-42ad-a7f6-fb2268433aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:22.296615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:22.352428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b36b0-6796-42ad-a7f6-fb2268433aa7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:22.458650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:22.458650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:22.458650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:22.458650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:22.460020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:22.574542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:22.587476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:22.588960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c95f1-aed9-4f84-9902-0129af263c7d service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:22.608032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 659/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:26.247935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ffbf9-154f-49d1-a446-b69237f9a152 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:26.248796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ffbf9-154f-49d1-a446-b69237f9a152 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:26.248796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ffbf9-154f-49d1-a446-b69237f9a152 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:26.250205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 664/1323] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:26.259418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c518948-56c4-413f-8418-4afb1ecc625b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:26.260253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c518948-56c4-413f-8418-4afb1ecc625b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:26.260438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c518948-56c4-413f-8418-4afb1ecc625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:26.351422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1c518948-56c4-413f-8418-4afb1ecc625b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:26.353618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1c518948-56c4-413f-8418-4afb1ecc625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:26.488605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 660/1324] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:26.526842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ffbf9-154f-49d1-a446-b69237f9a152 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:26.529826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-972ffbf9-154f-49d1-a446-b69237f9a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:26.558563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:26.671453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ffbf9-154f-49d1-a446-b69237f9a152 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:26.762210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:26.763240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:26.763673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:26.763977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:26.767163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:26.855291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:26.898019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:26.899945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0f72-0225-47c1-8660-e657416dbd68 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:26.932469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 665/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:29.445003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed0ea-9be1-4317-8db8-b689c2f11443 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:29.445813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed0ea-9be1-4317-8db8-b689c2f11443 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:29.445813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ed0ea-9be1-4317-8db8-b689c2f11443 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:29.447060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 661/1326] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:29.456173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2753-26e1-44cd-b7b2-65413b6599f2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:29.456812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2753-26e1-44cd-b7b2-65413b6599f2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:29.456954 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2753-26e1-44cd-b7b2-65413b6599f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:29.549416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a97a2753-26e1-44cd-b7b2-65413b6599f2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:29.550710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a97a2753-26e1-44cd-b7b2-65413b6599f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:29.679791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 666/1327] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:29.719322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ed0ea-9be1-4317-8db8-b689c2f11443 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:29.721452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d29ed0ea-9be1-4317-8db8-b689c2f11443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:29.795401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:29.854854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ed0ea-9be1-4317-8db8-b689c2f11443 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:29.963410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:29.963410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:29.963410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:29.964103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:29.965837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:30.097288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:30.100428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:30.101908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc3488-9107-4374-aba6-4687c62ecf32 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:30.121835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 662/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:32.851757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3e27-df34-4891-8acd-2528d8566843 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:32.852360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3e27-df34-4891-8acd-2528d8566843 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:32.852475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3e27-df34-4891-8acd-2528d8566843 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:32.854175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 667/1329] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:32.861369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d771753-37e2-4de8-ac88-574d8fe4df58 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:32.862140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d771753-37e2-4de8-ac88-574d8fe4df58 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:32.862140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d771753-37e2-4de8-ac88-574d8fe4df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:32.951166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9d771753-37e2-4de8-ac88-574d8fe4df58 None None] MFA Rules not processed for user `386db4044f5c4138b6858880d5c6bc71`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:32.952453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9d771753-37e2-4de8-ac88-574d8fe4df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:33.078129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 663/1330] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:33.118320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3e27-df34-4891-8acd-2528d8566843 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:33.120147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2eff3e27-df34-4891-8acd-2528d8566843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:33.153378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:33.241341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3e27-df34-4891-8acd-2528d8566843 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:33.326342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:33.326998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:33.327280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:33.327554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:33.329311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:33.420745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:33.458485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:33.460163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae21788-efd4-4644-848e-6ad7ab6d070a service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:33.481662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 668/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:40:36.522803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadb0ae-81ec-4252-9603-8d1508dd0799 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:36.523338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadb0ae-81ec-4252-9603-8d1508dd0799 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:36.523338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadb0ae-81ec-4252-9603-8d1508dd0799 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:36.524696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 664/1332] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:36.531220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2dcd-86ad-4da2-98c9-7ade7d8f2b20 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:36.531764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2dcd-86ad-4da2-98c9-7ade7d8f2b20 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:36.531764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2dcd-86ad-4da2-98c9-7ade7d8f2b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:36.607909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-3c9e2dcd-86ad-4da2-98c9-7ade7d8f2b20 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:36.609309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3c9e2dcd-86ad-4da2-98c9-7ade7d8f2b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:36.733200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 669/1333] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:36.780557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadb0ae-81ec-4252-9603-8d1508dd0799 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:36.782111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7fadb0ae-81ec-4252-9603-8d1508dd0799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:36.892432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:36.936261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadb0ae-81ec-4252-9603-8d1508dd0799 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:37.028616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:37.029384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:37.029747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:37.030103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:37.032136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:37.181750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:37.183663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44e13b-35a5-44b4-8ca3-4dc076aa6852 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:37.196979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:37.220514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 665/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:39.558316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba47ef-fbc7-42ac-a75b-7ea6ea307ddc None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:39.558886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba47ef-fbc7-42ac-a75b-7ea6ea307ddc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:39.558886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba47ef-fbc7-42ac-a75b-7ea6ea307ddc None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:39.560298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 670/1335] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:40:39.565926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efb8c7-b380-4561-8656-6cfa3f972704 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:39.567018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efb8c7-b380-4561-8656-6cfa3f972704 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:39.567113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efb8c7-b380-4561-8656-6cfa3f972704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:39.642786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e7efb8c7-b380-4561-8656-6cfa3f972704 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:39.644193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e7efb8c7-b380-4561-8656-6cfa3f972704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:39.798472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 666/1336] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:39.838941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba47ef-fbc7-42ac-a75b-7ea6ea307ddc None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:39.840485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ffba47ef-fbc7-42ac-a75b-7ea6ea307ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:39.891006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:39.957675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba47ef-fbc7-42ac-a75b-7ea6ea307ddc None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:40.034017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:40.034673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:40.034960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:40.035298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:40.037140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:40.133802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:40.160858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:40.162701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732bb8a-94b0-48b0-af1b-624c4b16ec0e service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:40.183379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 671/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:43.310598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fcb19-f311-445b-ae2d-e7cf7602a344 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:43.311161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fcb19-f311-445b-ae2d-e7cf7602a344 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:43.311161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fcb19-f311-445b-ae2d-e7cf7602a344 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:43.312609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 667/1338] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:43.319444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58ac84-5457-49c5-b3da-a4cf33622717 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:43.320281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58ac84-5457-49c5-b3da-a4cf33622717 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:43.320461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58ac84-5457-49c5-b3da-a4cf33622717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:43.403128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-bd58ac84-5457-49c5-b3da-a4cf33622717 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:43.404398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bd58ac84-5457-49c5-b3da-a4cf33622717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:43.541432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 672/1339] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:40:43.579892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fcb19-f311-445b-ae2d-e7cf7602a344 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:43.582281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-890fcb19-f311-445b-ae2d-e7cf7602a344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:43.676993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:43.693874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fcb19-f311-445b-ae2d-e7cf7602a344 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:43.697769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cafa5-38dc-4d3f-bfc9-0e0698303f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:43.698453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cafa5-38dc-4d3f-bfc9-0e0698303f74 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:43.698735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cafa5-38dc-4d3f-bfc9-0e0698303f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:43.699016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cafa5-38dc-4d3f-bfc9-0e0698303f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:43.710803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cafa5-38dc-4d3f-bfc9-0e0698303f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:43.712595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cafa5-38dc-4d3f-bfc9-0e0698303f74 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:43.723803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 668/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:40:48.706510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb76dd-251d-4a23-ae27-ffb027c82741 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:48.707817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb76dd-251d-4a23-ae27-ffb027c82741 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:48.707817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb76dd-251d-4a23-ae27-ffb027c82741 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:48.709466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 673/1341] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:48.715213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea5548-603a-4381-a373-e16095bdf9de None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:48.715751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea5548-603a-4381-a373-e16095bdf9de None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:48.715864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea5548-603a-4381-a373-e16095bdf9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:48.758789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-67ea5548-603a-4381-a373-e16095bdf9de None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:48.760110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-67ea5548-603a-4381-a373-e16095bdf9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:48.880554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 669/1342] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:40:48.924836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb76dd-251d-4a23-ae27-ffb027c82741 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:48.927032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5ebb76dd-251d-4a23-ae27-ffb027c82741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:48.941393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb76dd-251d-4a23-ae27-ffb027c82741 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:48.945141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0416-acb4-4516-b2eb-0a3add9641dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:48.945947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0416-acb4-4516-b2eb-0a3add9641dd service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:48.946069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0416-acb4-4516-b2eb-0a3add9641dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:48.946352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0416-acb4-4516-b2eb-0a3add9641dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:48.959566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0416-acb4-4516-b2eb-0a3add9641dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:48.961096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0416-acb4-4516-b2eb-0a3add9641dd service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:48.973104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 674/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:40:52.741325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec2368-62ea-4761-a35f-a21c950f8411 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:52.742013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec2368-62ea-4761-a35f-a21c950f8411 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:52.742181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec2368-62ea-4761-a35f-a21c950f8411 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:52.743781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 670/1344] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:40:52.750304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eca768-1aaf-4908-b2ff-46db86f38e53 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:52.750858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eca768-1aaf-4908-b2ff-46db86f38e53 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:52.751460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eca768-1aaf-4908-b2ff-46db86f38e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:52.776741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-63eca768-1aaf-4908-b2ff-46db86f38e53 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:52.777983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-63eca768-1aaf-4908-b2ff-46db86f38e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:52.786763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 675/1345] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:52.827573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec2368-62ea-4761-a35f-a21c950f8411 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:52.840226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec2368-62ea-4761-a35f-a21c950f8411 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:52.844466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6f61d-d424-4360-816c-273ead3408ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:52.845279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6f61d-d424-4360-816c-273ead3408ae service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:52.845648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6f61d-d424-4360-816c-273ead3408ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:52.846035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6f61d-d424-4360-816c-273ead3408ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:52.859547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6f61d-d424-4360-816c-273ead3408ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:52.862396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6f61d-d424-4360-816c-273ead3408ae service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:52.879979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 671/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:58.681632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d268fc-c69c-4b3d-8477-75e56b219dcb None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:58.682340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d268fc-c69c-4b3d-8477-75e56b219dcb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:58.682340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d268fc-c69c-4b3d-8477-75e56b219dcb None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:58.683606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 676/1347] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:40:58.691212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a24ea-c37a-4968-b5d0-46014488951c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:58.691874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a24ea-c37a-4968-b5d0-46014488951c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:58.692262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a24ea-c37a-4968-b5d0-46014488951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:58.713345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bd0a24ea-c37a-4968-b5d0-46014488951c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:58.714570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bd0a24ea-c37a-4968-b5d0-46014488951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:58.724139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 672/1348] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:58.762862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d268fc-c69c-4b3d-8477-75e56b219dcb None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:58.776073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d268fc-c69c-4b3d-8477-75e56b219dcb None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:58.778557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2dee2-c43e-4cb4-9057-74dead9566ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:58.779212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2dee2-c43e-4cb4-9057-74dead9566ed service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:58.779491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2dee2-c43e-4cb4-9057-74dead9566ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:58.779762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2dee2-c43e-4cb4-9057-74dead9566ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:58.791892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2dee2-c43e-4cb4-9057-74dead9566ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:58.793400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2dee2-c43e-4cb4-9057-74dead9566ed service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:58.804374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 677/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:41:05.019425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaaedd-0d36-49df-8f62-ec76d2e938a9 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:05.020333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaaedd-0d36-49df-8f62-ec76d2e938a9 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:05.020827 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaaedd-0d36-49df-8f62-ec76d2e938a9 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:05.022935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 673/1350] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:41:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:05.029124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bb637-58d8-4106-af08-b2d607c28dc7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:05.030103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bb637-58d8-4106-af08-b2d607c28dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:05.030556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835bb637-58d8-4106-af08-b2d607c28dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:05.056366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:05.151908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-835bb637-58d8-4106-af08-b2d607c28dc7 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:05.153414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-835bb637-58d8-4106-af08-b2d607c28dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:05.334496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 678/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:41:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:08.707681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=57 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:11.251765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23d82-a9a0-4b9d-8ea4-282e45b22954 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:11.253895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23d82-a9a0-4b9d-8ea4-282e45b22954 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:11.254758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23d82-a9a0-4b9d-8ea4-282e45b22954 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:11.257427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 674/1352] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:41:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:11.265915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644df61-032f-4a56-8d1b-5b2698a5b027 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:11.266721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644df61-032f-4a56-8d1b-5b2698a5b027 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:11.266721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644df61-032f-4a56-8d1b-5b2698a5b027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:11.308580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5644df61-032f-4a56-8d1b-5b2698a5b027 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:11.310989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5644df61-032f-4a56-8d1b-5b2698a5b027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:11.328724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 679/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:41:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:12.667835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07293d0-f739-4a4b-a662-2d6781980862 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:12.668418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07293d0-f739-4a4b-a662-2d6781980862 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:12.668918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07293d0-f739-4a4b-a662-2d6781980862 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:12.671060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 675/1354] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:41:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:12.677650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b811a83-3395-49d9-b7c3-2ea48bc20f7b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:12.678452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b811a83-3395-49d9-b7c3-2ea48bc20f7b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:12.678695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b811a83-3395-49d9-b7c3-2ea48bc20f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:12.700155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5b811a83-3395-49d9-b7c3-2ea48bc20f7b None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:12.701546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5b811a83-3395-49d9-b7c3-2ea48bc20f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:12.712050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 680/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:41:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:17.165543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62197bb-e285-4d1d-b97c-5803196b05c4 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:17.166081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62197bb-e285-4d1d-b97c-5803196b05c4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:17.166081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62197bb-e285-4d1d-b97c-5803196b05c4 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:17.167477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 676/1356] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:17.173586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea697421-c60a-477d-b429-9f5ae9e34188 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:17.174726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea697421-c60a-477d-b429-9f5ae9e34188 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:17.174726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea697421-c60a-477d-b429-9f5ae9e34188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:17.192978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ea697421-c60a-477d-b429-9f5ae9e34188 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:17.194183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ea697421-c60a-477d-b429-9f5ae9e34188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:17.214776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 681/1357] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:17.299870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578821-8bcb-4cd1-bdd3-9eaebc281176 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:17.300576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578821-8bcb-4cd1-bdd3-9eaebc281176 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:17.301296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578821-8bcb-4cd1-bdd3-9eaebc281176 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:17.308657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 677/1358] 10.222.0.58 () {54 vars in 961 bytes} [Fri May 27 03:41:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:17.315168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd9618-2cd0-4c43-bcdd-baa1b9d7e86c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:17.316156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd9618-2cd0-4c43-bcdd-baa1b9d7e86c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:17.316447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd9618-2cd0-4c43-bcdd-baa1b9d7e86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:17.397240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0dbd9618-2cd0-4c43-bcdd-baa1b9d7e86c None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:17.398674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0dbd9618-2cd0-4c43-bcdd-baa1b9d7e86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:17.496268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:17.528835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 682/1359] 10.222.0.58 () {58 vars in 1058 bytes} [Fri May 27 03:41:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:17.537987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00578821-8bcb-4cd1-bdd3-9eaebc281176 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:17.562678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00578821-8bcb-4cd1-bdd3-9eaebc281176 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:17.569812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbd527-cc00-4238-833e-b0875a9fc6fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:17.571027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd527-cc00-4238-833e-b0875a9fc6fc service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:17.571634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd527-cc00-4238-833e-b0875a9fc6fc service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:17.572136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbd527-cc00-4238-833e-b0875a9fc6fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:17.586033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbd527-cc00-4238-833e-b0875a9fc6fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:17.588204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbd527-cc00-4238-833e-b0875a9fc6fc service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:17.602750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 678/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:20.712488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ab10-b3db-4f0e-984d-646a5f5e8e32 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:20.713619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ab10-b3db-4f0e-984d-646a5f5e8e32 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:20.713619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ab10-b3db-4f0e-984d-646a5f5e8e32 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:20.714839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 683/1361] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:20.723605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82496bd3-b075-4bce-b1f3-b510f2f59319 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:20.724125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82496bd3-b075-4bce-b1f3-b510f2f59319 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:20.725698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82496bd3-b075-4bce-b1f3-b510f2f59319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:20.754462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-82496bd3-b075-4bce-b1f3-b510f2f59319 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:20.755419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-82496bd3-b075-4bce-b1f3-b510f2f59319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:20.765192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 679/1362] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:20.811933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26ab10-b3db-4f0e-984d-646a5f5e8e32 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:20.838548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26ab10-b3db-4f0e-984d-646a5f5e8e32 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:20.850928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889b7e8-7718-43f3-84e1-c2b66dc5a8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:20.852321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889b7e8-7718-43f3-84e1-c2b66dc5a8b1 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:20.856107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889b7e8-7718-43f3-84e1-c2b66dc5a8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:20.856607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889b7e8-7718-43f3-84e1-c2b66dc5a8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:20.885526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889b7e8-7718-43f3-84e1-c2b66dc5a8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:20.889622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889b7e8-7718-43f3-84e1-c2b66dc5a8b1 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:20.910446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 684/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:24.503267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3f99-556a-4059-a373-a8bf95872260 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:24.503927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3f99-556a-4059-a373-a8bf95872260 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:24.504091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc3f99-556a-4059-a373-a8bf95872260 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:24.505697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 680/1364] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:24.512094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587405e-28cd-4187-bf4f-c7eaadb2b49e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:24.512960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587405e-28cd-4187-bf4f-c7eaadb2b49e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:24.512960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587405e-28cd-4187-bf4f-c7eaadb2b49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:24.533068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1587405e-28cd-4187-bf4f-c7eaadb2b49e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:24.534296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1587405e-28cd-4187-bf4f-c7eaadb2b49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:24.542781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 685/1365] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:24.592738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa094bf-90cf-4f48-944f-359206cefc18 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:24.593266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa094bf-90cf-4f48-944f-359206cefc18 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:24.593828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa094bf-90cf-4f48-944f-359206cefc18 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:24.597093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 681/1366] 10.222.0.58 () {54 vars in 961 bytes} [Fri May 27 03:41:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:24.604811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff2f6e-bdff-4bce-bacf-b3a6cfc8a9f9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:24.608659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff2f6e-bdff-4bce-bacf-b3a6cfc8a9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:24.608659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff2f6e-bdff-4bce-bacf-b3a6cfc8a9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:24.623892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-85ff2f6e-bdff-4bce-bacf-b3a6cfc8a9f9 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:24.625147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-85ff2f6e-bdff-4bce-bacf-b3a6cfc8a9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:24.635520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 686/1367] 10.222.0.58 () {58 vars in 1058 bytes} [Fri May 27 03:41:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:24.645276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa094bf-90cf-4f48-944f-359206cefc18 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:24.680937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa094bf-90cf-4f48-944f-359206cefc18 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:24.692752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d564f7-4e94-4855-a401-e0c9939893dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:24.694154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d564f7-4e94-4855-a401-e0c9939893dc service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:24.694744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d564f7-4e94-4855-a401-e0c9939893dc service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:24.695389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d564f7-4e94-4855-a401-e0c9939893dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:24.710784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d564f7-4e94-4855-a401-e0c9939893dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:24.714420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d564f7-4e94-4855-a401-e0c9939893dc service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:24.734207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 682/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:41:27.912409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7939a-5df2-446f-937c-e2f8a5470a56 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:27.913135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7939a-5df2-446f-937c-e2f8a5470a56 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:27.913482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7939a-5df2-446f-937c-e2f8a5470a56 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:27.918315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 687/1369] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:27.925663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb4f04-f87c-455e-bfdc-5a2b8b4dd1d9 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:27.926210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb4f04-f87c-455e-bfdc-5a2b8b4dd1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:27.926640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb4f04-f87c-455e-bfdc-5a2b8b4dd1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:27.956857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-41fb4f04-f87c-455e-bfdc-5a2b8b4dd1d9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:27.958695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-41fb4f04-f87c-455e-bfdc-5a2b8b4dd1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:27.971697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 683/1370] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:28.160426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a167b16-4b05-4bed-9f01-f0b9039df769 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:28.160693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a167b16-4b05-4bed-9f01-f0b9039df769 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:28.161188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a167b16-4b05-4bed-9f01-f0b9039df769 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:28.162610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 688/1371] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:28.168397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf8605-16ab-4f66-95e8-20487b10e553 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:28.168743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf8605-16ab-4f66-95e8-20487b10e553 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:28.169026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf8605-16ab-4f66-95e8-20487b10e553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:28.192192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a5bf8605-16ab-4f66-95e8-20487b10e553 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:28.193627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5bf8605-16ab-4f66-95e8-20487b10e553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:28.202614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 684/1372] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:28.252900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a167b16-4b05-4bed-9f01-f0b9039df769 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:28.275863 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a167b16-4b05-4bed-9f01-f0b9039df769 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:28.285318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd052e9-0437-4c20-a1a4-858581226009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:28.286488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd052e9-0437-4c20-a1a4-858581226009 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:28.286992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd052e9-0437-4c20-a1a4-858581226009 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:28.287485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd052e9-0437-4c20-a1a4-858581226009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:28.301398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd052e9-0437-4c20-a1a4-858581226009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:28.303047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd052e9-0437-4c20-a1a4-858581226009 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:28.314801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 689/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:28.357450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31eada2-124a-492a-84aa-07f516390e2e None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:28.357450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31eada2-124a-492a-84aa-07f516390e2e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:28.357450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31eada2-124a-492a-84aa-07f516390e2e None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:28.358350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 685/1374] 10.222.0.58 () {54 vars in 963 bytes} [Fri May 27 03:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:28.366290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ffb9e-9810-4013-bec0-9aa7ea472f06 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:28.368299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ffb9e-9810-4013-bec0-9aa7ea472f06 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:28.368299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ffb9e-9810-4013-bec0-9aa7ea472f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:28.450610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-297ffb9e-9810-4013-bec0-9aa7ea472f06 None None] MFA Rules not processed for user `cd4fb121b8de406899d5698c84d0903b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:28.451947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-297ffb9e-9810-4013-bec0-9aa7ea472f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:28.554414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 690/1375] 10.222.0.58 () {58 vars in 1060 bytes} [Fri May 27 03:41:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:28.566643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31eada2-124a-492a-84aa-07f516390e2e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:28.581339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31eada2-124a-492a-84aa-07f516390e2e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:28.586586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718a9ef-b28d-4dd4-945c-8e02c3d49b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:28.587627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a9ef-b28d-4dd4-945c-8e02c3d49b32 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:28.588191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a9ef-b28d-4dd4-945c-8e02c3d49b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:28.588976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718a9ef-b28d-4dd4-945c-8e02c3d49b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:28.602986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718a9ef-b28d-4dd4-945c-8e02c3d49b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:28.605588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718a9ef-b28d-4dd4-945c-8e02c3d49b32 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:28.621951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 686/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:30.904220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df8cd-7528-40e7-a262-5b7061d44db4 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:30.904832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df8cd-7528-40e7-a262-5b7061d44db4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:30.905411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92df8cd-7528-40e7-a262-5b7061d44db4 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:30.907360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 691/1377] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:30.913584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a300e7-28ca-4acf-90c6-8a732ffe0e8d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:30.914161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a300e7-28ca-4acf-90c6-8a732ffe0e8d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:30.914161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a300e7-28ca-4acf-90c6-8a732ffe0e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:30.935720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c8a300e7-28ca-4acf-90c6-8a732ffe0e8d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:30.936862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c8a300e7-28ca-4acf-90c6-8a732ffe0e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:30.947715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 687/1378] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:31.006873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df8cd-7528-40e7-a262-5b7061d44db4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:31.031886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92df8cd-7528-40e7-a262-5b7061d44db4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:31.038970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509be080-36b4-4005-982d-4c0a857c4ef1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:31.039635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be080-36b4-4005-982d-4c0a857c4ef1 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:31.039915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be080-36b4-4005-982d-4c0a857c4ef1 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:31.040196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be080-36b4-4005-982d-4c0a857c4ef1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:31.050627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509be080-36b4-4005-982d-4c0a857c4ef1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:31.052230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509be080-36b4-4005-982d-4c0a857c4ef1 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:31.070626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 692/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:33.673353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9aec3-af9e-4e56-87b8-2105a7ca0c7d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:33.674543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9aec3-af9e-4e56-87b8-2105a7ca0c7d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:33.674853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9aec3-af9e-4e56-87b8-2105a7ca0c7d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:33.676303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 688/1380] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:41:33.682066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa490352-b58e-4fac-b650-27fb4e008327 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:33.682758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa490352-b58e-4fac-b650-27fb4e008327 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:33.682758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa490352-b58e-4fac-b650-27fb4e008327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:33.703669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fa490352-b58e-4fac-b650-27fb4e008327 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:33.706137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fa490352-b58e-4fac-b650-27fb4e008327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:33.715550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 693/1381] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:33.771342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9aec3-af9e-4e56-87b8-2105a7ca0c7d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:33.787381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9aec3-af9e-4e56-87b8-2105a7ca0c7d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:33.793078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9676e-3788-4b0d-9633-7eed9b8cc072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:33.793670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9676e-3788-4b0d-9633-7eed9b8cc072 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:33.793950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9676e-3788-4b0d-9633-7eed9b8cc072 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:33.794224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9676e-3788-4b0d-9633-7eed9b8cc072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:33.803789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9676e-3788-4b0d-9633-7eed9b8cc072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:33.805649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9676e-3788-4b0d-9633-7eed9b8cc072 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:33.817732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 689/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:36.324273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8f6c-b2aa-4c38-9fd1-f7b9a639b73d None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:36.324888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8f6c-b2aa-4c38-9fd1-f7b9a639b73d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:36.324888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e8f6c-b2aa-4c38-9fd1-f7b9a639b73d None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:36.326240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 694/1383] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:36.333269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d203-7162-4b4a-8141-1e4b1f0555b5 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:36.335077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d203-7162-4b4a-8141-1e4b1f0555b5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:36.335692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d203-7162-4b4a-8141-1e4b1f0555b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:36.361492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cf86d203-7162-4b4a-8141-1e4b1f0555b5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:36.363284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cf86d203-7162-4b4a-8141-1e4b1f0555b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:36.373746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 690/1384] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:36.420452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8f6c-b2aa-4c38-9fd1-f7b9a639b73d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:36.442439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e8f6c-b2aa-4c38-9fd1-f7b9a639b73d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:36.447462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b9f7b-df93-49b2-8a34-507ba46f3d5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:36.448201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b9f7b-df93-49b2-8a34-507ba46f3d5b service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:36.448614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b9f7b-df93-49b2-8a34-507ba46f3d5b service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:36.448967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b9f7b-df93-49b2-8a34-507ba46f3d5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:36.459148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b9f7b-df93-49b2-8a34-507ba46f3d5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:36.461116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b9f7b-df93-49b2-8a34-507ba46f3d5b service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:36.472780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 695/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:39.130246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0517a9-c575-4286-907f-f462dbfc45b0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:39.131021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0517a9-c575-4286-907f-f462dbfc45b0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:39.131021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0517a9-c575-4286-907f-f462dbfc45b0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:39.132145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 691/1386] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:39.138674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15988dd-ae0e-457c-b928-a05ab407489f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:39.139199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15988dd-ae0e-457c-b928-a05ab407489f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:39.139199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15988dd-ae0e-457c-b928-a05ab407489f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:39.160102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b15988dd-ae0e-457c-b928-a05ab407489f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:39.161396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b15988dd-ae0e-457c-b928-a05ab407489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:39.169696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 696/1387] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:39.215797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0517a9-c575-4286-907f-f462dbfc45b0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:39.224310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=40 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:39.231905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0517a9-c575-4286-907f-f462dbfc45b0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:39.238483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53de57-019f-4d16-9229-5fab5af07a96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:39.241262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53de57-019f-4d16-9229-5fab5af07a96 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:39.241262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53de57-019f-4d16-9229-5fab5af07a96 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:39.241262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53de57-019f-4d16-9229-5fab5af07a96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:39.252448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53de57-019f-4d16-9229-5fab5af07a96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:39.254672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53de57-019f-4d16-9229-5fab5af07a96 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:39.267477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 692/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:41.781327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf9b6-c8d8-4624-875a-908f961fa63e None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:41.781327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf9b6-c8d8-4624-875a-908f961fa63e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:41.781327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf9b6-c8d8-4624-875a-908f961fa63e None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:41.787880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 697/1389] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:41.788995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a4abf-0ccd-46f4-a596-be4063cd3c9f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:41.789485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a4abf-0ccd-46f4-a596-be4063cd3c9f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:41.789575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a4abf-0ccd-46f4-a596-be4063cd3c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:41.808869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9d4a4abf-0ccd-46f4-a596-be4063cd3c9f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:41.810053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9d4a4abf-0ccd-46f4-a596-be4063cd3c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:41.820030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 693/1390] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:41.877892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cf9b6-c8d8-4624-875a-908f961fa63e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:41.893575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cf9b6-c8d8-4624-875a-908f961fa63e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:41.899006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bf806-d644-4bfd-b681-c25fd91833b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:41.899652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf806-d644-4bfd-b681-c25fd91833b7 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:41.899937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf806-d644-4bfd-b681-c25fd91833b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:41.900212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf806-d644-4bfd-b681-c25fd91833b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:41.909255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bf806-d644-4bfd-b681-c25fd91833b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:41.910838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bf806-d644-4bfd-b681-c25fd91833b7 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:41.921258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 698/1391] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:44.552364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8aad2-3125-4eff-afbc-0050b3ef0874 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:44.552969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8aad2-3125-4eff-afbc-0050b3ef0874 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:44.552969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8aad2-3125-4eff-afbc-0050b3ef0874 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:44.554260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 694/1392] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:44.559470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a8b7-c241-402a-85da-cbfc358a5c53 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:44.559904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a8b7-c241-402a-85da-cbfc358a5c53 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:44.560040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a8b7-c241-402a-85da-cbfc358a5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:44.579246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-9271a8b7-c241-402a-85da-cbfc358a5c53 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:44.580640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9271a8b7-c241-402a-85da-cbfc358a5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:44.590449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 699/1393] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:44.636104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8aad2-3125-4eff-afbc-0050b3ef0874 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:44.651687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8aad2-3125-4eff-afbc-0050b3ef0874 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:44.658056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2d0d8-6869-4165-8417-936fa4e524b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:44.658647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d0d8-6869-4165-8417-936fa4e524b9 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:44.658934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d0d8-6869-4165-8417-936fa4e524b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:44.659216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d0d8-6869-4165-8417-936fa4e524b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:44.668234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2d0d8-6869-4165-8417-936fa4e524b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:44.669896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2d0d8-6869-4165-8417-936fa4e524b9 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:44.680486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 695/1394] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:45.724045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6de1e3-3a91-4a1f-88e5-99fefcf2d565 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:45.724803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6de1e3-3a91-4a1f-88e5-99fefcf2d565 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:45.724803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6de1e3-3a91-4a1f-88e5-99fefcf2d565 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:45.726376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 700/1395] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:45.733996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f15308-1059-4ef1-aafd-4e86d43c6eec None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:45.735367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f15308-1059-4ef1-aafd-4e86d43c6eec None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:45.735367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f15308-1059-4ef1-aafd-4e86d43c6eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:45.756137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-94f15308-1059-4ef1-aafd-4e86d43c6eec None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:45.757429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94f15308-1059-4ef1-aafd-4e86d43c6eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:45.765934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 696/1396] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:45.824908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff84b3-292f-4b7f-aa50-dca35c82e33d None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:45.825338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff84b3-292f-4b7f-aa50-dca35c82e33d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:45.825605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff84b3-292f-4b7f-aa50-dca35c82e33d None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:45.827185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 701/1397] 10.222.0.58 () {54 vars in 962 bytes} [Fri May 27 03:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:45.833421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b24f4-8c9f-4d53-bc15-5fa176b1a783 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:45.833797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b24f4-8c9f-4d53-bc15-5fa176b1a783 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:45.834076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b24f4-8c9f-4d53-bc15-5fa176b1a783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:45.886578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f22b24f4-8c9f-4d53-bc15-5fa176b1a783 None None] MFA Rules not processed for user `dc6f327db350476e9e5194719da4cbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:45.887826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f22b24f4-8c9f-4d53-bc15-5fa176b1a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:45.948874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 697/1398] 10.222.0.58 () {58 vars in 1059 bytes} [Fri May 27 03:41:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:45.956705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff84b3-292f-4b7f-aa50-dca35c82e33d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:45.967516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff84b3-292f-4b7f-aa50-dca35c82e33d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:45.971385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2a138-aa3f-4fd2-adfa-5890c7d5d774 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:45.972135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2a138-aa3f-4fd2-adfa-5890c7d5d774 service glance] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:45.972491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2a138-aa3f-4fd2-adfa-5890c7d5d774 service glance] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:45.972800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2a138-aa3f-4fd2-adfa-5890c7d5d774 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:45.985475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2a138-aa3f-4fd2-adfa-5890c7d5d774 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:45.987838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2a138-aa3f-4fd2-adfa-5890c7d5d774 service glance] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:45.998344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 702/1399] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:46.749141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91733b1-4404-4ea3-bf5a-467b9a7496fc None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:46.750617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91733b1-4404-4ea3-bf5a-467b9a7496fc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:46.750617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91733b1-4404-4ea3-bf5a-467b9a7496fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:46.815029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b91733b1-4404-4ea3-bf5a-467b9a7496fc None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:46.816388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b91733b1-4404-4ea3-bf5a-467b9a7496fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:46.900541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 698/1400] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:41:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:46.946483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d38583-8088-4cdf-bd91-6574b65cf566 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:46.947069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d38583-8088-4cdf-bd91-6574b65cf566 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:46.947069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d38583-8088-4cdf-bd91-6574b65cf566 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:46.948455 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 703/1401] 10.222.0.58 () {54 vars in 942 bytes} [Fri May 27 03:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:46.956108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c186d-1fe0-4636-b412-61024d9fcf4d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:46.956399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c186d-1fe0-4636-b412-61024d9fcf4d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:46.956825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c186d-1fe0-4636-b412-61024d9fcf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:46.976278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:47.011312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-003c186d-1fe0-4636-b412-61024d9fcf4d None None] MFA Rules not processed for user `1c8eaac98b1b4f5db6c0b60c3f01f711`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:47.013109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-003c186d-1fe0-4636-b412-61024d9fcf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:47.092918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 699/1402] 10.222.0.58 () {58 vars in 1039 bytes} [Fri May 27 03:41:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:47.102898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d38583-8088-4cdf-bd91-6574b65cf566 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:47.118687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d38583-8088-4cdf-bd91-6574b65cf566 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:47.119620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29f17-58a9-4ef1-b999-452fdc133e2d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:47.119940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29f17-58a9-4ef1-b999-452fdc133e2d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:47.120744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b29f17-58a9-4ef1-b999-452fdc133e2d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:47.121860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 700/1403] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:41:47.125105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:47.127027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:47.127388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:47.127742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:47.137833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9082187-e667-4dc7-9938-ba8085ad732c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:47.138117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9082187-e667-4dc7-9938-ba8085ad732c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:47.138408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9082187-e667-4dc7-9938-ba8085ad732c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:47.138561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:47.140180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:47.147254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:47.152034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 704/1404] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:41:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:41:47.166682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a9082187-e667-4dc7-9938-ba8085ad732c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:47.166682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a9082187-e667-4dc7-9938-ba8085ad732c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:47.168393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 701/1405] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:47.217396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:47.230955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c3dd1-9dc4-41c2-829f-7cd24f9be9fb service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:47.237850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c21d-487e-436d-886d-67ac7e8ae289 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:47.238520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c21d-487e-436d-886d-67ac7e8ae289 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:47.238831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c21d-487e-436d-886d-67ac7e8ae289 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:47.244018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c21d-487e-436d-886d-67ac7e8ae289 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:47.248637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021c21d-487e-436d-886d-67ac7e8ae289 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:47.250822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021c21d-487e-436d-886d-67ac7e8ae289 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:47.262174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 705/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:49.975316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618ac4c-8b74-4dc4-b97e-3b67f58d8fd0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:49.976165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618ac4c-8b74-4dc4-b97e-3b67f58d8fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:49.976165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618ac4c-8b74-4dc4-b97e-3b67f58d8fd0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:49.980951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 702/1407] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:41:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:49.984235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8b9b-1b7a-4c69-9ad6-2c320f5f3e67 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:49.984781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8b9b-1b7a-4c69-9ad6-2c320f5f3e67 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:49.986269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8b9b-1b7a-4c69-9ad6-2c320f5f3e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:50.010795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e97b8b9b-1b7a-4c69-9ad6-2c320f5f3e67 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:50.015288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e97b8b9b-1b7a-4c69-9ad6-2c320f5f3e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:50.021918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 706/1408] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:50.071965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618ac4c-8b74-4dc4-b97e-3b67f58d8fd0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:50.089554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618ac4c-8b74-4dc4-b97e-3b67f58d8fd0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:50.096292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c5117-34cc-4af9-a7d1-ac346ab9f077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:50.097004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c5117-34cc-4af9-a7d1-ac346ab9f077 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:50.097303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c5117-34cc-4af9-a7d1-ac346ab9f077 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:50.097598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c5117-34cc-4af9-a7d1-ac346ab9f077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:50.109235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c5117-34cc-4af9-a7d1-ac346ab9f077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:50.111306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c5117-34cc-4af9-a7d1-ac346ab9f077 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:50.131936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 703/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:00.004887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=9 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:00.309432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=48 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:12.078423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a09ee-6c94-4647-a1ad-131b11f42db7 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:12.078994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a09ee-6c94-4647-a1ad-131b11f42db7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:12.078994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a09ee-6c94-4647-a1ad-131b11f42db7 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:12.081341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 707/1410] 10.222.0.58 () {54 vars in 918 bytes} [Fri May 27 03:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:12.086476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d19cd-eb6f-420f-997c-e54308c619c8 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:12.086932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d19cd-eb6f-420f-997c-e54308c619c8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:12.087041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d19cd-eb6f-420f-997c-e54308c619c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:12.105871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cb5d19cd-eb6f-420f-997c-e54308c619c8 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:12.107054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cb5d19cd-eb6f-420f-997c-e54308c619c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:12.115410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 704/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Fri May 27 03:42:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:12.130043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474fcf-b726-4e49-8cae-847fb63a899e None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:12.130342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474fcf-b726-4e49-8cae-847fb63a899e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:12.130628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81474fcf-b726-4e49-8cae-847fb63a899e None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:12.132044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 708/1412] 10.222.0.58 () {54 vars in 918 bytes} [Fri May 27 03:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:12.136674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8da87-cebb-4319-9e0c-278c1ee778d3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:12.136961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8da87-cebb-4319-9e0c-278c1ee778d3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:12.137236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8da87-cebb-4319-9e0c-278c1ee778d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:12.155454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-dfe8da87-cebb-4319-9e0c-278c1ee778d3 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:12.156674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-dfe8da87-cebb-4319-9e0c-278c1ee778d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:12.166486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 705/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Fri May 27 03:42:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:14.919581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64c90b-16cf-4f5d-88ac-34236fb7e95b None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:14.920167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64c90b-16cf-4f5d-88ac-34236fb7e95b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:14.920167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64c90b-16cf-4f5d-88ac-34236fb7e95b None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:14.921573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:14.927947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3589-6a34-47ff-94fa-bfaba8d05810 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:14.928483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3589-6a34-47ff-94fa-bfaba8d05810 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:14.928483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb3589-6a34-47ff-94fa-bfaba8d05810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:14.947578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-defb3589-6a34-47ff-94fa-bfaba8d05810 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:14.948843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-defb3589-6a34-47ff-94fa-bfaba8d05810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:14.957836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:15.003619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64c90b-16cf-4f5d-88ac-34236fb7e95b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:15.018641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64c90b-16cf-4f5d-88ac-34236fb7e95b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:15.023883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dbf84-cfe9-4608-95e7-0ff68ba48717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:15.024524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dbf84-cfe9-4608-95e7-0ff68ba48717 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:15.024828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dbf84-cfe9-4608-95e7-0ff68ba48717 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:15.025101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dbf84-cfe9-4608-95e7-0ff68ba48717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:15.034233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23dbf84-cfe9-4608-95e7-0ff68ba48717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:15.035838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23dbf84-cfe9-4608-95e7-0ff68ba48717 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:15.045623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 710/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:16.036064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfc43-17f3-44c0-8d28-37fae7a9a1f9 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:16.036693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfc43-17f3-44c0-8d28-37fae7a9a1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:16.036693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dfc43-17f3-44c0-8d28-37fae7a9a1f9 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:16.038280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:42:16.044248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7fb77-e1db-40ba-a6f0-b4af2456f69f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:16.044788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7fb77-e1db-40ba-a6f0-b4af2456f69f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:16.044903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7fb77-e1db-40ba-a6f0-b4af2456f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:16.067757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-2dc7fb77-e1db-40ba-a6f0-b4af2456f69f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:16.069033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2dc7fb77-e1db-40ba-a6f0-b4af2456f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:16.078437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:16.244806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dfc43-17f3-44c0-8d28-37fae7a9a1f9 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:16.259534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dfc43-17f3-44c0-8d28-37fae7a9a1f9 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:16.265293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5af0b-c572-4710-93fc-31ddfadb5669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:16.265941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5af0b-c572-4710-93fc-31ddfadb5669 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:16.268064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5af0b-c572-4710-93fc-31ddfadb5669 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:16.268064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5af0b-c572-4710-93fc-31ddfadb5669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:16.275447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5af0b-c572-4710-93fc-31ddfadb5669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:16.277252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5af0b-c572-4710-93fc-31ddfadb5669 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:16.287879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 708/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:17.230384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e2565-afe7-4cb6-8beb-31702beaf79b None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:17.232898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e2565-afe7-4cb6-8beb-31702beaf79b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:17.232898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e2565-afe7-4cb6-8beb-31702beaf79b None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:17.232898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:17.238814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af57c6b-d366-4b50-8593-93364d42658f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:17.239638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af57c6b-d366-4b50-8593-93364d42658f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:17.239844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af57c6b-d366-4b50-8593-93364d42658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:17.260616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6af57c6b-d366-4b50-8593-93364d42658f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:17.261858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6af57c6b-d366-4b50-8593-93364d42658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:17.270356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:17.427887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e2565-afe7-4cb6-8beb-31702beaf79b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:17.445113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e2565-afe7-4cb6-8beb-31702beaf79b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:17.452056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31739e4b-871d-420a-98c7-f47368fbe54c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:17.452764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31739e4b-871d-420a-98c7-f47368fbe54c service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:17.453058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31739e4b-871d-420a-98c7-f47368fbe54c service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:17.453341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31739e4b-871d-420a-98c7-f47368fbe54c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:17.463001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31739e4b-871d-420a-98c7-f47368fbe54c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:17.464645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31739e4b-871d-420a-98c7-f47368fbe54c service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:17.479001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 713/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:18.414342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2781d-f66f-43a0-b04d-37f91ebbab38 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:18.414882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2781d-f66f-43a0-b04d-37f91ebbab38 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:18.414882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2781d-f66f-43a0-b04d-37f91ebbab38 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:18.416311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:18.422876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa880c-64c4-46d2-85dc-74cbee910b67 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:18.423378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa880c-64c4-46d2-85dc-74cbee910b67 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:18.423664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa880c-64c4-46d2-85dc-74cbee910b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:18.442911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-2baa880c-64c4-46d2-85dc-74cbee910b67 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:18.444123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2baa880c-64c4-46d2-85dc-74cbee910b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:18.451924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:18.607700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2781d-f66f-43a0-b04d-37f91ebbab38 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:18.622237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2781d-f66f-43a0-b04d-37f91ebbab38 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:18.628225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4101-97de-40d6-882f-db766c75ab5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:18.628883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4101-97de-40d6-882f-db766c75ab5a service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:18.629161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4101-97de-40d6-882f-db766c75ab5a service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:18.629434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4101-97de-40d6-882f-db766c75ab5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:18.637896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4101-97de-40d6-882f-db766c75ab5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:18.639512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd4101-97de-40d6-882f-db766c75ab5a service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:18.650044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 711/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:19.603174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f512-d8a0-4ef1-a380-1a2312a63046 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:19.603746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f512-d8a0-4ef1-a380-1a2312a63046 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:19.603746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384f512-d8a0-4ef1-a380-1a2312a63046 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:19.605150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:19.610974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb0e9e-0f45-47ee-b3e5-c46a37796c21 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:19.611432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb0e9e-0f45-47ee-b3e5-c46a37796c21 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:19.611538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb0e9e-0f45-47ee-b3e5-c46a37796c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:19.631007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2edb0e9e-0f45-47ee-b3e5-c46a37796c21 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:19.632223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2edb0e9e-0f45-47ee-b3e5-c46a37796c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:19.640116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:19.793831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384f512-d8a0-4ef1-a380-1a2312a63046 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:19.809385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384f512-d8a0-4ef1-a380-1a2312a63046 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:19.816073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae7b7f-3125-4cf7-a659-a56a1029c9be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:19.816758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7b7f-3125-4cf7-a659-a56a1029c9be service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:19.817048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7b7f-3125-4cf7-a659-a56a1029c9be service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:19.817329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7b7f-3125-4cf7-a659-a56a1029c9be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:19.826046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae7b7f-3125-4cf7-a659-a56a1029c9be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:19.827643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae7b7f-3125-4cf7-a659-a56a1029c9be service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:19.837638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 716/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:20.773221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88c8cb-9891-4613-8779-3334ed805e89 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:20.773802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88c8cb-9891-4613-8779-3334ed805e89 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:20.774049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88c8cb-9891-4613-8779-3334ed805e89 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:20.776525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:20.782139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5befb-ed92-41ae-9a6a-f303a792a63a None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:20.782742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5befb-ed92-41ae-9a6a-f303a792a63a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:20.782742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5befb-ed92-41ae-9a6a-f303a792a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:20.801531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-dce5befb-ed92-41ae-9a6a-f303a792a63a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:20.802820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-dce5befb-ed92-41ae-9a6a-f303a792a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:20.811008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:20.966432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88c8cb-9891-4613-8779-3334ed805e89 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:20.981354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88c8cb-9891-4613-8779-3334ed805e89 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:20.987372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e673e-7751-443c-a2e0-c50778d52edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:20.988050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e673e-7751-443c-a2e0-c50778d52edd service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:20.988367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e673e-7751-443c-a2e0-c50778d52edd service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:20.988694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e673e-7751-443c-a2e0-c50778d52edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:20.997583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e673e-7751-443c-a2e0-c50778d52edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:20.999192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e673e-7751-443c-a2e0-c50778d52edd service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:21.009852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 714/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:21.946927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee365b6-baea-46d2-97a4-5ca6252488f7 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:21.947494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee365b6-baea-46d2-97a4-5ca6252488f7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:21.947494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee365b6-baea-46d2-97a4-5ca6252488f7 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:21.948883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:21.954074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84f22b-3d32-463f-88f2-ce5fc13804c5 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:21.954758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84f22b-3d32-463f-88f2-ce5fc13804c5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:21.955154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84f22b-3d32-463f-88f2-ce5fc13804c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:21.973806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4e84f22b-3d32-463f-88f2-ce5fc13804c5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:21.975047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4e84f22b-3d32-463f-88f2-ce5fc13804c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:21.983439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:22.135525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee365b6-baea-46d2-97a4-5ca6252488f7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:22.149799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee365b6-baea-46d2-97a4-5ca6252488f7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:22.155187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cae0-83d6-41c5-bbfe-9f00f4577303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:22.155834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cae0-83d6-41c5-bbfe-9f00f4577303 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:22.156116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cae0-83d6-41c5-bbfe-9f00f4577303 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:22.156396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cae0-83d6-41c5-bbfe-9f00f4577303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:22.165174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cae0-83d6-41c5-bbfe-9f00f4577303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:22.166768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cae0-83d6-41c5-bbfe-9f00f4577303 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:22.178065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 719/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:23.124610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186e279-f606-4754-a626-0d3e40a93fe7 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:23.125229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186e279-f606-4754-a626-0d3e40a93fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:23.125229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186e279-f606-4754-a626-0d3e40a93fe7 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:23.127258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:23.133522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e193be-6a86-4380-a224-6c916a09cf20 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:23.134021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e193be-6a86-4380-a224-6c916a09cf20 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:23.134021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e193be-6a86-4380-a224-6c916a09cf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:23.153707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c5e193be-6a86-4380-a224-6c916a09cf20 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:23.154966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c5e193be-6a86-4380-a224-6c916a09cf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:23.164938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:23.319631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186e279-f606-4754-a626-0d3e40a93fe7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:23.337907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186e279-f606-4754-a626-0d3e40a93fe7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:23.343528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e535acc-a021-4b0c-b68c-4cc330c6fe08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:23.344210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e535acc-a021-4b0c-b68c-4cc330c6fe08 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:23.344468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e535acc-a021-4b0c-b68c-4cc330c6fe08 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:23.344770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e535acc-a021-4b0c-b68c-4cc330c6fe08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:23.355929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e535acc-a021-4b0c-b68c-4cc330c6fe08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:23.359171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e535acc-a021-4b0c-b68c-4cc330c6fe08 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:23.374458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 717/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:24.314671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f1ba-58f7-4557-a442-e80656016ac5 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:24.315230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f1ba-58f7-4557-a442-e80656016ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:24.315230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f1ba-58f7-4557-a442-e80656016ac5 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:24.316691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:24.322585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645995d4-3e25-442e-833e-c3a65c77af31 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:24.323037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645995d4-3e25-442e-833e-c3a65c77af31 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:24.323163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645995d4-3e25-442e-833e-c3a65c77af31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:24.341983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-645995d4-3e25-442e-833e-c3a65c77af31 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:24.346572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-645995d4-3e25-442e-833e-c3a65c77af31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:24.351687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:24.505557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f1ba-58f7-4557-a442-e80656016ac5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:24.521554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f1ba-58f7-4557-a442-e80656016ac5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:24.528412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cd7da-f06d-4ba5-8132-9e95bb301212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:24.529190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd7da-f06d-4ba5-8132-9e95bb301212 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:24.529510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd7da-f06d-4ba5-8132-9e95bb301212 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:24.529794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd7da-f06d-4ba5-8132-9e95bb301212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:24.540009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cd7da-f06d-4ba5-8132-9e95bb301212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:24.541746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cd7da-f06d-4ba5-8132-9e95bb301212 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:24.554422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 722/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:26.648891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2a14-9d20-4c56-8d25-d2b0c6472b4a None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:26.649450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2a14-9d20-4c56-8d25-d2b0c6472b4a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:26.649450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f2a14-9d20-4c56-8d25-d2b0c6472b4a None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:26.650715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:26.656270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aba18f-f0f5-4204-b50f-09aee18d5187 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:26.656723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aba18f-f0f5-4204-b50f-09aee18d5187 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:26.656879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aba18f-f0f5-4204-b50f-09aee18d5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:26.675068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-37aba18f-f0f5-4204-b50f-09aee18d5187 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:26.676323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-37aba18f-f0f5-4204-b50f-09aee18d5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:26.684598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:26.723936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f2a14-9d20-4c56-8d25-d2b0c6472b4a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:26.734736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f2a14-9d20-4c56-8d25-d2b0c6472b4a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:26.738246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e41b3-1413-4d87-a6ad-d2d21c422cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:26.738884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e41b3-1413-4d87-a6ad-d2d21c422cae service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:26.739165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e41b3-1413-4d87-a6ad-d2d21c422cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:26.739438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e41b3-1413-4d87-a6ad-d2d21c422cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:26.748602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e41b3-1413-4d87-a6ad-d2d21c422cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:26.750067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e41b3-1413-4d87-a6ad-d2d21c422cae service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:26.760943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 720/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:29.145848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f8c94-e777-4b99-9d65-df27a1190597 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.146363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f8c94-e777-4b99-9d65-df27a1190597 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.146363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f8c94-e777-4b99-9d65-df27a1190597 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.147701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:29.153105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4dce-c7fd-4190-b85e-94fb6377528d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.153528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4dce-c7fd-4190-b85e-94fb6377528d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.153694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4dce-c7fd-4190-b85e-94fb6377528d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.171726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-53ad4dce-c7fd-4190-b85e-94fb6377528d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:29.173977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-53ad4dce-c7fd-4190-b85e-94fb6377528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:29.181529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:29.219220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f8c94-e777-4b99-9d65-df27a1190597 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.231322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f8c94-e777-4b99-9d65-df27a1190597 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.234857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae069e-0415-4103-9fac-cf1954f5b6f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.236514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae069e-0415-4103-9fac-cf1954f5b6f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.236830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae069e-0415-4103-9fac-cf1954f5b6f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.237118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae069e-0415-4103-9fac-cf1954f5b6f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.246220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae069e-0415-4103-9fac-cf1954f5b6f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.247651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae069e-0415-4103-9fac-cf1954f5b6f1 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.257583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 725/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:31.862499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c201dcb-6327-4cb0-bc78-71f1b503b9a2 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:31.863217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c201dcb-6327-4cb0-bc78-71f1b503b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:31.863217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c201dcb-6327-4cb0-bc78-71f1b503b9a2 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:31.864477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:42:31.870119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5d61e-aeda-41b2-8c68-d0028ce09a23 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:31.870596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5d61e-aeda-41b2-8c68-d0028ce09a23 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:31.870710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5d61e-aeda-41b2-8c68-d0028ce09a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:31.889208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-38c5d61e-aeda-41b2-8c68-d0028ce09a23 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:31.890414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-38c5d61e-aeda-41b2-8c68-d0028ce09a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:31.898610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:31.936289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c201dcb-6327-4cb0-bc78-71f1b503b9a2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:31.947709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c201dcb-6327-4cb0-bc78-71f1b503b9a2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:31.951198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95ffd5-021a-4abf-b2f2-0c251be20ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:31.951824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ffd5-021a-4abf-b2f2-0c251be20ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:31.952109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ffd5-021a-4abf-b2f2-0c251be20ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:31.952385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95ffd5-021a-4abf-b2f2-0c251be20ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:31.961567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95ffd5-021a-4abf-b2f2-0c251be20ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:31.962998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95ffd5-021a-4abf-b2f2-0c251be20ef4 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:31.972363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 723/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:34.313933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dddeff-fa5d-4960-824c-47c5a58fe936 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:34.314441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dddeff-fa5d-4960-824c-47c5a58fe936 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:34.314549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dddeff-fa5d-4960-824c-47c5a58fe936 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:34.315876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:34.321208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8ca14-b61b-4c3e-bdd7-8c386f206218 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:34.321657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8ca14-b61b-4c3e-bdd7-8c386f206218 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:34.321763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8ca14-b61b-4c3e-bdd7-8c386f206218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:34.339345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-87d8ca14-b61b-4c3e-bdd7-8c386f206218 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:34.342813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-87d8ca14-b61b-4c3e-bdd7-8c386f206218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:34.349068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:34.387121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dddeff-fa5d-4960-824c-47c5a58fe936 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:34.397834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dddeff-fa5d-4960-824c-47c5a58fe936 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:34.401417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9775c-4eba-46c0-a1ac-33988ac93fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:34.402052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9775c-4eba-46c0-a1ac-33988ac93fab service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:34.402338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9775c-4eba-46c0-a1ac-33988ac93fab service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:34.402614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9775c-4eba-46c0-a1ac-33988ac93fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:34.412098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9775c-4eba-46c0-a1ac-33988ac93fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:34.413558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9775c-4eba-46c0-a1ac-33988ac93fab service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:34.423434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 728/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:36.701466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcfac-f7f1-4a52-bf80-2d4d74c1f4e6 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.702167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcfac-f7f1-4a52-bf80-2d4d74c1f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.702167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbcfac-f7f1-4a52-bf80-2d4d74c1f4e6 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.703441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:36.709404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10312-1e7a-4aa6-b10b-d4ba89405322 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.709885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10312-1e7a-4aa6-b10b-d4ba89405322 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.709993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a10312-1e7a-4aa6-b10b-d4ba89405322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.728022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-63a10312-1e7a-4aa6-b10b-d4ba89405322 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:36.729241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-63a10312-1e7a-4aa6-b10b-d4ba89405322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:36.737733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:36.775217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcfac-f7f1-4a52-bf80-2d4d74c1f4e6 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.787746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbcfac-f7f1-4a52-bf80-2d4d74c1f4e6 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.791397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67824ffc-0e28-4f7e-bdb1-5dedfdcb228d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.792042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824ffc-0e28-4f7e-bdb1-5dedfdcb228d service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.792326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824ffc-0e28-4f7e-bdb1-5dedfdcb228d service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.792687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824ffc-0e28-4f7e-bdb1-5dedfdcb228d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.803455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67824ffc-0e28-4f7e-bdb1-5dedfdcb228d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.805057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67824ffc-0e28-4f7e-bdb1-5dedfdcb228d service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.815590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 726/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:39.177037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807159f-a8c2-4c41-a063-7e2bc83e4877 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:39.177622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807159f-a8c2-4c41-a063-7e2bc83e4877 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:39.177622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807159f-a8c2-4c41-a063-7e2bc83e4877 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:39.178975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:39.184429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63b46-21f4-4e11-aeea-d5da7f31a797 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:39.184892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63b46-21f4-4e11-aeea-d5da7f31a797 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:39.185024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63b46-21f4-4e11-aeea-d5da7f31a797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:39.209712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3ff63b46-21f4-4e11-aeea-d5da7f31a797 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:39.210918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3ff63b46-21f4-4e11-aeea-d5da7f31a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:39.221851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:39.259595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807159f-a8c2-4c41-a063-7e2bc83e4877 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:39.270934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807159f-a8c2-4c41-a063-7e2bc83e4877 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:39.274395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a2b63-9e80-4c6c-a34b-b3a49ed7baa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:39.275026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a2b63-9e80-4c6c-a34b-b3a49ed7baa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:39.275311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a2b63-9e80-4c6c-a34b-b3a49ed7baa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:39.275588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a2b63-9e80-4c6c-a34b-b3a49ed7baa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:39.285805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a2b63-9e80-4c6c-a34b-b3a49ed7baa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:39.287272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a2b63-9e80-4c6c-a34b-b3a49ed7baa2 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:39.296658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 731/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:41.626987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fdaf5-2b4d-4eaa-823c-8db9e7288a88 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.627546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fdaf5-2b4d-4eaa-823c-8db9e7288a88 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.627546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fdaf5-2b4d-4eaa-823c-8db9e7288a88 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.628981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:41.634484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cad00d-107f-4737-93a8-9c3c50b7adea None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.634910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cad00d-107f-4737-93a8-9c3c50b7adea None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.635041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cad00d-107f-4737-93a8-9c3c50b7adea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.641929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=70 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:41.654268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-51cad00d-107f-4737-93a8-9c3c50b7adea None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:41.661609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-51cad00d-107f-4737-93a8-9c3c50b7adea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:41.664225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:41.701377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fdaf5-2b4d-4eaa-823c-8db9e7288a88 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.707185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=73 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:41.713141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fdaf5-2b4d-4eaa-823c-8db9e7288a88 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.716572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43839afc-e8d4-452e-9d13-268b8f453d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.717205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43839afc-e8d4-452e-9d13-268b8f453d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.717483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43839afc-e8d4-452e-9d13-268b8f453d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.717801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43839afc-e8d4-452e-9d13-268b8f453d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.727923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43839afc-e8d4-452e-9d13-268b8f453d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.729397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43839afc-e8d4-452e-9d13-268b8f453d5f service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.739191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 729/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:44.078609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f5d6-46e0-44b7-a38c-18fb91b47863 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:44.079257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f5d6-46e0-44b7-a38c-18fb91b47863 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:44.079257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f5d6-46e0-44b7-a38c-18fb91b47863 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:44.080721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:44.085984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71028d58-012b-4777-9dc1-82657dced99c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:44.086406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71028d58-012b-4777-9dc1-82657dced99c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:44.086548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71028d58-012b-4777-9dc1-82657dced99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:44.105445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-71028d58-012b-4777-9dc1-82657dced99c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:44.106660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-71028d58-012b-4777-9dc1-82657dced99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:44.115317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:44.152456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121f5d6-46e0-44b7-a38c-18fb91b47863 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:44.163938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121f5d6-46e0-44b7-a38c-18fb91b47863 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:44.167533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81db4-f11b-4b50-b61c-860eceb43c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:44.168180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81db4-f11b-4b50-b61c-860eceb43c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:44.168469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81db4-f11b-4b50-b61c-860eceb43c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:44.168803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81db4-f11b-4b50-b61c-860eceb43c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:44.179440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e81db4-f11b-4b50-b61c-860eceb43c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:44.180929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e81db4-f11b-4b50-b61c-860eceb43c5f service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:44.191418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 734/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:46.469240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455fae5-b630-49b7-b3de-ef751e739b02 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:46.469892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455fae5-b630-49b7-b3de-ef751e739b02 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:46.469892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455fae5-b630-49b7-b3de-ef751e739b02 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:46.471240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:46.477763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1928ce-0445-4113-9656-a67a19f16174 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:46.478520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1928ce-0445-4113-9656-a67a19f16174 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:46.478520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1928ce-0445-4113-9656-a67a19f16174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:46.498534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-9d1928ce-0445-4113-9656-a67a19f16174 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:46.500014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9d1928ce-0445-4113-9656-a67a19f16174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:46.509189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:46.548875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455fae5-b630-49b7-b3de-ef751e739b02 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:46.559522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455fae5-b630-49b7-b3de-ef751e739b02 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:46.563472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6233681-9bea-4c31-ad77-08214c9c5002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:46.564110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6233681-9bea-4c31-ad77-08214c9c5002 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:46.564399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6233681-9bea-4c31-ad77-08214c9c5002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:46.564718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6233681-9bea-4c31-ad77-08214c9c5002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:46.573883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6233681-9bea-4c31-ad77-08214c9c5002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:46.575316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6233681-9bea-4c31-ad77-08214c9c5002 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:46.585291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 732/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:48.830947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182aa0c5-e0e0-40a0-82c1-16bd1045cca1 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:48.831518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182aa0c5-e0e0-40a0-82c1-16bd1045cca1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:48.831518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182aa0c5-e0e0-40a0-82c1-16bd1045cca1 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:48.832895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:42:48.838285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda2bf5-5e8a-496e-bd9e-fdabe41c6e8f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:48.838929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda2bf5-5e8a-496e-bd9e-fdabe41c6e8f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:48.838929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda2bf5-5e8a-496e-bd9e-fdabe41c6e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:48.857002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-dbda2bf5-5e8a-496e-bd9e-fdabe41c6e8f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:48.858205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-dbda2bf5-5e8a-496e-bd9e-fdabe41c6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:48.867230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:48.905101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182aa0c5-e0e0-40a0-82c1-16bd1045cca1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:48.916348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182aa0c5-e0e0-40a0-82c1-16bd1045cca1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:48.922337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2341b5-07cd-4c6f-8cbc-575dcdd97404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:48.922972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2341b5-07cd-4c6f-8cbc-575dcdd97404 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:48.923252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2341b5-07cd-4c6f-8cbc-575dcdd97404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:48.923528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2341b5-07cd-4c6f-8cbc-575dcdd97404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:48.934318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2341b5-07cd-4c6f-8cbc-575dcdd97404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:48.935747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2341b5-07cd-4c6f-8cbc-575dcdd97404 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:48.946066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 737/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:58.851781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=17 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:58.941863 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=17 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:22.812095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bb163-d568-4375-909d-23c982a577f1 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.812831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bb163-d568-4375-909d-23c982a577f1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.812831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bb163-d568-4375-909d-23c982a577f1 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.814228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 734/1471] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:22.820648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ac4e4-0ec7-4a2e-ab3e-41343348abf8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.821162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ac4e4-0ec7-4a2e-ab3e-41343348abf8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.821380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ac4e4-0ec7-4a2e-ab3e-41343348abf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.840352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0f3ac4e4-0ec7-4a2e-ab3e-41343348abf8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:22.841681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0f3ac4e4-0ec7-4a2e-ab3e-41343348abf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:22.850286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 738/1472] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:22.902650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8a95-740a-4d6a-85ba-8a69235d84c5 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.902943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8a95-740a-4d6a-85ba-8a69235d84c5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.903227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de8a95-740a-4d6a-85ba-8a69235d84c5 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.905227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 735/1473] 10.222.0.58 () {54 vars in 962 bytes} [Fri May 27 03:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:22.910726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc55ce-5528-439e-99b6-a8d517c20fdd None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.911022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc55ce-5528-439e-99b6-a8d517c20fdd None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.911311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc55ce-5528-439e-99b6-a8d517c20fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.930369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-dddc55ce-5528-439e-99b6-a8d517c20fdd None None] MFA Rules not processed for user `dc6f327db350476e9e5194719da4cbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:22.931595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-dddc55ce-5528-439e-99b6-a8d517c20fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:22.938965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 739/1474] 10.222.0.58 () {58 vars in 1059 bytes} [Fri May 27 03:43:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:22.946596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8a95-740a-4d6a-85ba-8a69235d84c5 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:22.959583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de8a95-740a-4d6a-85ba-8a69235d84c5 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:22.963055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d87f4f-91cd-4c4b-b28d-9e57d71a06f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:22.963722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87f4f-91cd-4c4b-b28d-9e57d71a06f2 service glance] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.964026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87f4f-91cd-4c4b-b28d-9e57d71a06f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.964301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87f4f-91cd-4c4b-b28d-9e57d71a06f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.974780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d87f4f-91cd-4c4b-b28d-9e57d71a06f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:22.976336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d87f4f-91cd-4c4b-b28d-9e57d71a06f2 service glance] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:22.987943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 736/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:43:25.446707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb47bcf-93b7-4bda-b710-73c34fd2c38f None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.447279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb47bcf-93b7-4bda-b710-73c34fd2c38f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.447279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb47bcf-93b7-4bda-b710-73c34fd2c38f None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.448692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 740/1476] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:25.454574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c7566-42de-4b96-8c8b-91d33af08424 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.455013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c7566-42de-4b96-8c8b-91d33af08424 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.455139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c7566-42de-4b96-8c8b-91d33af08424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.473923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-140c7566-42de-4b96-8c8b-91d33af08424 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:25.475118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-140c7566-42de-4b96-8c8b-91d33af08424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:25.483153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 737/1477] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:25.506741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb47bcf-93b7-4bda-b710-73c34fd2c38f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.520239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb47bcf-93b7-4bda-b710-73c34fd2c38f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.524666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.525318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.525603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.525880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.526608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.528360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:43:25.528360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:43:25.528360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:43:25.530105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250396f-ba71-4dbb-95cf-4778b7b9bac9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.549932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 741/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Fri May 27 03:43:25 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:27.791778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6023d-c0c8-4f9f-98f2-bee68e199d68 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:27.792351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6023d-c0c8-4f9f-98f2-bee68e199d68 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:27.792351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6023d-c0c8-4f9f-98f2-bee68e199d68 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:27.793712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 738/1479] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:27.799682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb95b0-f21d-4022-a735-3175e7b570bf None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:27.800113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb95b0-f21d-4022-a735-3175e7b570bf None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:27.800249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb95b0-f21d-4022-a735-3175e7b570bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:27.820205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-49eb95b0-f21d-4022-a735-3175e7b570bf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:27.821481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-49eb95b0-f21d-4022-a735-3175e7b570bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:27.830400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 742/1480] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:27.877713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6023d-c0c8-4f9f-98f2-bee68e199d68 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:27.894567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6023d-c0c8-4f9f-98f2-bee68e199d68 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:27.900142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984794e-934a-49f2-837a-dda9e878dbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:27.900825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984794e-934a-49f2-837a-dda9e878dbb0 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:27.901114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984794e-934a-49f2-837a-dda9e878dbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:27.901391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984794e-934a-49f2-837a-dda9e878dbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:27.911874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984794e-934a-49f2-837a-dda9e878dbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:27.913602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984794e-934a-49f2-837a-dda9e878dbb0 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:27.924533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 739/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:30.410787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843ed59-0328-44de-a44c-491d515ff009 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:30.411348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843ed59-0328-44de-a44c-491d515ff009 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:30.411348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843ed59-0328-44de-a44c-491d515ff009 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:30.412749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 743/1482] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:30.418370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0173da-a930-41ba-b756-86a6a08e14b1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:30.418835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0173da-a930-41ba-b756-86a6a08e14b1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:30.418948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0173da-a930-41ba-b756-86a6a08e14b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:30.436884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6f0173da-a930-41ba-b756-86a6a08e14b1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:30.438074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6f0173da-a930-41ba-b756-86a6a08e14b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:30.445909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 740/1483] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:30.483869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843ed59-0328-44de-a44c-491d515ff009 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:30.494853 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843ed59-0328-44de-a44c-491d515ff009 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:30.499081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ffe20-03ff-4743-b914-164906b6a225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:30.499936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ffe20-03ff-4743-b914-164906b6a225 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:30.500226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ffe20-03ff-4743-b914-164906b6a225 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:30.500528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ffe20-03ff-4743-b914-164906b6a225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:30.511052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ffe20-03ff-4743-b914-164906b6a225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:30.514355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ffe20-03ff-4743-b914-164906b6a225 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:30.524818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 744/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:43:32.882139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cec8e-a553-4ec8-9d1d-4a1f8b6cbdc4 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:32.882833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cec8e-a553-4ec8-9d1d-4a1f8b6cbdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:32.882833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cec8e-a553-4ec8-9d1d-4a1f8b6cbdc4 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:32.884061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 741/1485] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:32.889903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157b8d-e65d-4495-9e3a-7914ca41709e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:32.890506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157b8d-e65d-4495-9e3a-7914ca41709e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:32.890506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157b8d-e65d-4495-9e3a-7914ca41709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:32.909914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-aa157b8d-e65d-4495-9e3a-7914ca41709e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:32.911145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-aa157b8d-e65d-4495-9e3a-7914ca41709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:32.920174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 745/1486] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:32.959999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cec8e-a553-4ec8-9d1d-4a1f8b6cbdc4 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:32.972016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cec8e-a553-4ec8-9d1d-4a1f8b6cbdc4 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:32.975781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdfe92-e69d-4ddc-a2d2-9a502178d448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:32.976468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdfe92-e69d-4ddc-a2d2-9a502178d448 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:32.976788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdfe92-e69d-4ddc-a2d2-9a502178d448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:32.977066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdfe92-e69d-4ddc-a2d2-9a502178d448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:32.987147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdfe92-e69d-4ddc-a2d2-9a502178d448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:32.989131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefdfe92-e69d-4ddc-a2d2-9a502178d448 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:32.998223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 742/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:35.428548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236351e7-70b2-4964-a57d-6ef9db4d20c3 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:35.429128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236351e7-70b2-4964-a57d-6ef9db4d20c3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:35.429128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236351e7-70b2-4964-a57d-6ef9db4d20c3 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:35.430526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 746/1488] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:35.436708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08c78-c55e-4d10-9219-0e835619c7cd None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:35.437284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08c78-c55e-4d10-9219-0e835619c7cd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:35.437284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a08c78-c55e-4d10-9219-0e835619c7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:35.459458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-74a08c78-c55e-4d10-9219-0e835619c7cd None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:35.461669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-74a08c78-c55e-4d10-9219-0e835619c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:35.472886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 743/1489] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:35.514902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236351e7-70b2-4964-a57d-6ef9db4d20c3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:35.526824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236351e7-70b2-4964-a57d-6ef9db4d20c3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:35.531634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df44a7-3d93-4601-9f72-e9b8050e7f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:35.532288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df44a7-3d93-4601-9f72-e9b8050e7f47 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:35.532596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df44a7-3d93-4601-9f72-e9b8050e7f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:35.532876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df44a7-3d93-4601-9f72-e9b8050e7f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:35.543544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df44a7-3d93-4601-9f72-e9b8050e7f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:35.545206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df44a7-3d93-4601-9f72-e9b8050e7f47 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:35.556362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 747/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:43:38.255154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbbe05-ce2d-4992-ab12-66dce35dced2 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:38.255813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbbe05-ce2d-4992-ab12-66dce35dced2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:38.255813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbbe05-ce2d-4992-ab12-66dce35dced2 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:38.257408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 744/1491] 10.222.0.58 () {54 vars in 926 bytes} [Fri May 27 03:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:43:38.265209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbcef-8a60-4717-8683-9c650c192351 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:38.265209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbcef-8a60-4717-8683-9c650c192351 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:38.265209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bbcef-8a60-4717-8683-9c650c192351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:38.288279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b69bbcef-8a60-4717-8683-9c650c192351 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:38.289556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b69bbcef-8a60-4717-8683-9c650c192351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:38.298380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 748/1492] 10.222.0.58 () {58 vars in 1023 bytes} [Fri May 27 03:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:38.338381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbbe05-ce2d-4992-ab12-66dce35dced2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:38.350211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbbe05-ce2d-4992-ab12-66dce35dced2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:38.353799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b6b8a-3c6b-4d12-85bb-b21f1592b005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:38.354447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b6b8a-3c6b-4d12-85bb-b21f1592b005 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:38.354731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b6b8a-3c6b-4d12-85bb-b21f1592b005 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:38.355011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b6b8a-3c6b-4d12-85bb-b21f1592b005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:38.365594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b6b8a-3c6b-4d12-85bb-b21f1592b005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:38.367145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b6b8a-3c6b-4d12-85bb-b21f1592b005 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:38.377920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 745/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:48.282597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=33 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:48.372421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=33 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:30.856474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:30.857144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:30.857390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:30.875630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:45:30.876882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:30.884982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 749/1494] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:45:30.894829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:30.895336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:30.895452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:30.914842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:45:30.916051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:30.924519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 746/1495] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:45:30.929173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:30.941138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc523e-9ab5-4d5c-8d82-6434805b2d15 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:30.946122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:30.946785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:30.947073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:30.947351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:30.948080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:45:30.948388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:30.949468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:30.949468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:45:30.949468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:45:30.950460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:30.962877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 750/1496] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:45:30.971449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:30.980762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4be50-199f-4bd8-8a1b-922d7877cd42 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:30.986070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:30.986722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:30.987008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:30.987307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:30.987977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:30.989797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:30.989797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:45:30.989797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:45:30.991002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:31.102668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 747/1497] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:45:30 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:45:31.108180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:31.118010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174c383-2472-4835-a90d-0c75a0b4570e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:31.180349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:31.181072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.181371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.181641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.182377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:31.183424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:31.183424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:45:31.183424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:45:31.184711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:31.260941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 751/1498] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:45:31 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:45:31.269682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:31.280718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3128ebb-275c-489e-ab4f-74c91b6c69c5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:31.285227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:31.285889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.286191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.286449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.287203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:31.288216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:31.288216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:45:31.288216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:45:31.289140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:31.302545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 748/1499] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:45:31.306359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:31.316237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022757-7995-4a73-9743-e60cd644e51f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:31.320811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:31.321472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.321755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.322040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.322766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:31.323777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:31.323777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:45:31.323777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:45:31.324699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc458ab6-f294-4516-a26f-9250180cb796 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:31.337928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 752/1500] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:45:31.342912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:31.352884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bef45-e81e-4262-840f-9da2cf15e876 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:31.358116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:31.359048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.359270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.359544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] PATH_INFO: `/v3/projects/3373410431db40948c2e7814135ed83e/users/8ccc822f77454739a78061355fb8725f/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.374215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3373410431db40948c2e7814135ed83e, user_id=8ccc822f77454739a78061355fb8725f, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:31.375248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:31.375248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:45:31.375248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:45:31.377216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f32ad-3b67-4d16-9d7f-44d4c2496ca2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:31.443383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 749/1501] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:45:31 2022] PUT /identity/v3/projects/3373410431db40948c2e7814135ed83e/users/8ccc822f77454739a78061355fb8725f/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:45:31.455032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119f746-66e6-4156-a487-bc94a00b76ae None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.455324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119f746-66e6-4156-a487-bc94a00b76ae None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.455601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119f746-66e6-4156-a487-bc94a00b76ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.481924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b119f746-66e6-4156-a487-bc94a00b76ae None None] MFA Rules not processed for user `8ccc822f77454739a78061355fb8725f`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:45:31.483192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b119f746-66e6-4156-a487-bc94a00b76ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:31.553881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 753/1502] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:45:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:45:31.567237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050f47-0510-4794-a8f6-48a16114c215 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.567559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050f47-0510-4794-a8f6-48a16114c215 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.567843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050f47-0510-4794-a8f6-48a16114c215 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.569238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 750/1503] 10.222.0.58 () {54 vars in 963 bytes} [Fri May 27 03:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:45:31.574228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90def6c-132c-4519-ab98-16ec0e5dc78e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.574529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90def6c-132c-4519-ab98-16ec0e5dc78e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.574815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90def6c-132c-4519-ab98-16ec0e5dc78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.594208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a90def6c-132c-4519-ab98-16ec0e5dc78e None None] MFA Rules not processed for user `cd4fb121b8de406899d5698c84d0903b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:45:31.595149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a90def6c-132c-4519-ab98-16ec0e5dc78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:31.638883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:31.646871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 754/1504] 10.222.0.58 () {58 vars in 1060 bytes} [Fri May 27 03:45:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:45:31.655334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050f47-0510-4794-a8f6-48a16114c215 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:31.666193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050f47-0510-4794-a8f6-48a16114c215 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:31.670187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:31.670817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.671103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.671417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.680050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:31.681516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:31.690483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 751/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:45:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:45:31.735454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113028d9-93f8-4fa9-8bfa-ab27eef23113 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.735748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113028d9-93f8-4fa9-8bfa-ab27eef23113 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.736032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113028d9-93f8-4fa9-8bfa-ab27eef23113 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.737367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 755/1506] 10.222.0.58 () {52 vars in 829 bytes} [Fri May 27 03:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:45:31.741114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:31.808151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51486e59-5a84-4512-89e9-2662b31b9a60 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:31.813865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:31.814884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:31.815175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:31.815452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] PATH_INFO: `/v3/users/8ccc822f77454739a78061355fb8725f` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:31.816749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccc822f77454739a78061355fb8725f)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:31.817786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:31.817786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:45:31.817786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:45:31.819012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:32.038614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccc822f77454739a78061355fb8725f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:32.039700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccc822f77454739a78061355fb8725f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:32.052977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccc822f77454739a78061355fb8725f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:32.053653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccc822f77454739a78061355fb8725f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:32.117460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccc822f77454739a78061355fb8725f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:32.137615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:32.147831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 752/1507] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:45:31 2022] DELETE /identity/v3/users/8ccc822f77454739a78061355fb8725f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:45:32.153003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:32.153700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:32.153805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:32.175285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:45:32.176564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:32.242726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 756/1508] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:45:32.253340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:32.254783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:32.318864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9060db8-aa7c-4677-9376-ee3b67dbbe51 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:32.322349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fbc22-3ace-4893-bff9-f758b40c9588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:32.329729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fbc22-3ace-4893-bff9-f758b40c9588 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:32.329729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fbc22-3ace-4893-bff9-f758b40c9588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:32.329729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fbc22-3ace-4893-bff9-f758b40c9588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:32.339962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fbc22-3ace-4893-bff9-f758b40c9588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:32.343053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fbc22-3ace-4893-bff9-f758b40c9588 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:32.359561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 753/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:45:32.864023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:32.865510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:32.883049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1837a5-ee20-490b-bc60-d5cbd3d15a27 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:32.888737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:32.889617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:32.889901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:32.890734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] PATH_INFO: `/v3/projects/3373410431db40948c2e7814135ed83e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:32.891377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3373410431db40948c2e7814135ed83e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:32.892428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:45:32.892428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:45:32.892428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:45:32.893689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:33.014226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3373410431db40948c2e7814135ed83e', 'request_id': 'req-895e48a3-f154-41d8-82b1-71250bc213fb'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:33.077684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3373410431db40948c2e7814135ed83e', 'request_id': 'req-895e48a3-f154-41d8-82b1-71250bc213fb'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:33.092576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3373410431db40948c2e7814135ed83e', 'request_id': 'req-895e48a3-f154-41d8-82b1-71250bc213fb'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:33.103618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] The token cache is being invalidate because project 3373410431db40948c2e7814135ed83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:45:33.103908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:33.104691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-895e48a3-f154-41d8-82b1-71250bc213fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:33.106260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 757/1510] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:45:32 2022] DELETE /identity/v3/projects/3373410431db40948c2e7814135ed83e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:45:42.353707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:43.100612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:16.449700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98625911-977e-4a81-8a6e-8af13ed962d0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:16.450438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98625911-977e-4a81-8a6e-8af13ed962d0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:16.450438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98625911-977e-4a81-8a6e-8af13ed962d0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:16.451637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:16.457089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687f7c-9d3a-4ba2-94d5-ca02f24cebc0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:16.457617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687f7c-9d3a-4ba2-94d5-ca02f24cebc0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:16.457711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687f7c-9d3a-4ba2-94d5-ca02f24cebc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:16.478176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c4687f7c-9d3a-4ba2-94d5-ca02f24cebc0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:16.479407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c4687f7c-9d3a-4ba2-94d5-ca02f24cebc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:16.544065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:16.731805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98625911-977e-4a81-8a6e-8af13ed962d0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:16.733682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-98625911-977e-4a81-8a6e-8af13ed962d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:16.813197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98625911-977e-4a81-8a6e-8af13ed962d0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:16.819332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56291c2c-94d9-4aa3-accb-adf5f7cc2899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:16.819985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291c2c-94d9-4aa3-accb-adf5f7cc2899 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:16.820272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291c2c-94d9-4aa3-accb-adf5f7cc2899 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:16.820579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291c2c-94d9-4aa3-accb-adf5f7cc2899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:16.828727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56291c2c-94d9-4aa3-accb-adf5f7cc2899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:16.830359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56291c2c-94d9-4aa3-accb-adf5f7cc2899 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:16.841272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 755/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:17.763082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5327c2-41d6-4872-8aef-056c0f2ca06e None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:17.763610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5327c2-41d6-4872-8aef-056c0f2ca06e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:17.763610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5327c2-41d6-4872-8aef-056c0f2ca06e None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:17.764975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:17.770245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:17.770740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:17.770859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:17.785539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476fe5-1996-4d40-9e7d-29b9aad4c4a6 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:17.785909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476fe5-1996-4d40-9e7d-29b9aad4c4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:17.786275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476fe5-1996-4d40-9e7d-29b9aad4c4a6 None None] PATH_INFO: `/v3` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:17.788037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 760/1515] 10.222.0.44 () {54 vars in 925 bytes} [Fri May 27 03:47:17 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:47:17.789041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:17.790239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:17.793967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c031eb-b340-48c5-80f5-a25911852009 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:17.794302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c031eb-b340-48c5-80f5-a25911852009 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:17.794643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c031eb-b340-48c5-80f5-a25911852009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:17.798191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 756/1516] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:17.816142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d8c031eb-b340-48c5-80f5-a25911852009 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:17.817396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d8c031eb-b340-48c5-80f5-a25911852009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:17.887354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 761/1517] 10.222.0.44 () {58 vars in 1015 bytes} [Fri May 27 03:47:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:47:17.949519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:17.962019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a09f57-8b1e-43a0-b9ac-7cb6ec195a64 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:17.967802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6265fdc-e7cd-43a8-bf9b-f4c83633d51c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:17.968452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6265fdc-e7cd-43a8-bf9b-f4c83633d51c service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:17.968767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6265fdc-e7cd-43a8-bf9b-f4c83633d51c service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:17.969048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6265fdc-e7cd-43a8-bf9b-f4c83633d51c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:17.978062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6265fdc-e7cd-43a8-bf9b-f4c83633d51c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:17.979649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6265fdc-e7cd-43a8-bf9b-f4c83633d51c service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:17.989133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 757/1518] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:22.060623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae92f-2431-447d-9c17-c5b9c405249a None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:22.061185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae92f-2431-447d-9c17-c5b9c405249a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:22.061185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae92f-2431-447d-9c17-c5b9c405249a None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:22.062485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 762/1519] 10.222.0.58 () {54 vars in 960 bytes} [Fri May 27 03:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:22.067895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae810be5-8b31-40f5-8142-fe8083cf26d1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:22.068972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae810be5-8b31-40f5-8142-fe8083cf26d1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:22.069248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae810be5-8b31-40f5-8142-fe8083cf26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:22.089489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ae810be5-8b31-40f5-8142-fe8083cf26d1 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:22.090706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ae810be5-8b31-40f5-8142-fe8083cf26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:22.106491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 758/1520] 10.222.0.58 () {58 vars in 1057 bytes} [Fri May 27 03:47:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:22.114376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae92f-2431-447d-9c17-c5b9c405249a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:22.128790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae92f-2431-447d-9c17-c5b9c405249a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:22.135294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bc61b-37c0-4856-a137-823fcf1d890b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:22.135961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bc61b-37c0-4856-a137-823fcf1d890b service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:22.136254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bc61b-37c0-4856-a137-823fcf1d890b service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:22.136570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bc61b-37c0-4856-a137-823fcf1d890b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:22.145727 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bc61b-37c0-4856-a137-823fcf1d890b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:22.147652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bc61b-37c0-4856-a137-823fcf1d890b service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:22.157767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 763/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:29.306883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed821f28-78b1-4d6c-b472-61613aabf37d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:29.307472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed821f28-78b1-4d6c-b472-61613aabf37d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:29.307472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed821f28-78b1-4d6c-b472-61613aabf37d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:29.308873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:29.314290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a0980-d0f7-4c00-93f4-1a0e9c60a8d4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:29.314804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a0980-d0f7-4c00-93f4-1a0e9c60a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:29.314919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a0980-d0f7-4c00-93f4-1a0e9c60a8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:29.334901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1c6a0980-d0f7-4c00-93f4-1a0e9c60a8d4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:29.336129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1c6a0980-d0f7-4c00-93f4-1a0e9c60a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:29.348227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:29.516558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed821f28-78b1-4d6c-b472-61613aabf37d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:29.532017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed821f28-78b1-4d6c-b472-61613aabf37d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:29.539147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6f3d1-ef2c-4ef4-a8cb-903a1eb70858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:29.539820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6f3d1-ef2c-4ef4-a8cb-903a1eb70858 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:29.540123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6f3d1-ef2c-4ef4-a8cb-903a1eb70858 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:29.540399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6f3d1-ef2c-4ef4-a8cb-903a1eb70858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:29.549533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6f3d1-ef2c-4ef4-a8cb-903a1eb70858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:29.551214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6f3d1-ef2c-4ef4-a8cb-903a1eb70858 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:29.562528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 760/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:30.457085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26200f00-2c9d-4635-ba90-c1d5ca62e8a2 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:30.457791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26200f00-2c9d-4635-ba90-c1d5ca62e8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:30.457791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26200f00-2c9d-4635-ba90-c1d5ca62e8a2 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:30.460172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:30.465753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2773d2-848a-4fc2-8f01-0b2396cfee3f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:30.466205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2773d2-848a-4fc2-8f01-0b2396cfee3f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:30.466325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2773d2-848a-4fc2-8f01-0b2396cfee3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:30.485704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0b2773d2-848a-4fc2-8f01-0b2396cfee3f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:30.486944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0b2773d2-848a-4fc2-8f01-0b2396cfee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:30.495755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:30.650671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26200f00-2c9d-4635-ba90-c1d5ca62e8a2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:30.665920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26200f00-2c9d-4635-ba90-c1d5ca62e8a2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:30.671419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6f28a-851a-498a-bae5-bdd4e7660141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:30.672057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f28a-851a-498a-bae5-bdd4e7660141 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:30.672366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f28a-851a-498a-bae5-bdd4e7660141 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:30.672670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f28a-851a-498a-bae5-bdd4e7660141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:30.681484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f28a-851a-498a-bae5-bdd4e7660141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:30.683036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f28a-851a-498a-bae5-bdd4e7660141 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:30.692939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 766/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:34.985478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac8d85-fc65-4bf7-b13c-21d713d524bd None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:34.986078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac8d85-fc65-4bf7-b13c-21d713d524bd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:34.986078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac8d85-fc65-4bf7-b13c-21d713d524bd None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:34.987420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:34.993582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ac36-1681-4995-a2a6-550f94ef04d8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:34.994444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ac36-1681-4995-a2a6-550f94ef04d8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:34.994444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0ac36-1681-4995-a2a6-550f94ef04d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:35.013612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-06f0ac36-1681-4995-a2a6-550f94ef04d8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:35.014850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-06f0ac36-1681-4995-a2a6-550f94ef04d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:35.024388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:47:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:35.061700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac8d85-fc65-4bf7-b13c-21d713d524bd None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:35.063540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9fac8d85-fc65-4bf7-b13c-21d713d524bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:35.129804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac8d85-fc65-4bf7-b13c-21d713d524bd None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:35.134217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43070-a831-47db-a905-bdb15d3279c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:35.134820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43070-a831-47db-a905-bdb15d3279c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:35.135104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43070-a831-47db-a905-bdb15d3279c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:35.135380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43070-a831-47db-a905-bdb15d3279c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:35.145737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43070-a831-47db-a905-bdb15d3279c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:35.147878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43070-a831-47db-a905-bdb15d3279c4 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:35.159138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 763/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:47:37.460551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd9bc1-73e7-4ff9-b936-aa1591b7cc14 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:37.461190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd9bc1-73e7-4ff9-b936-aa1591b7cc14 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:37.461190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd9bc1-73e7-4ff9-b936-aa1591b7cc14 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:37.462800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:47:37.468914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31780-6bc1-4c4d-b795-6a610003e935 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:37.469470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31780-6bc1-4c4d-b795-6a610003e935 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:37.469470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf31780-6bc1-4c4d-b795-6a610003e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:37.489562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9bf31780-6bc1-4c4d-b795-6a610003e935 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:37.491056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9bf31780-6bc1-4c4d-b795-6a610003e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:37.499157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:37.542594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd9bc1-73e7-4ff9-b936-aa1591b7cc14 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:37.553934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd9bc1-73e7-4ff9-b936-aa1591b7cc14 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:37.557687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a674b8-c81a-4579-a439-41663c1af283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:37.558367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a674b8-c81a-4579-a439-41663c1af283 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:37.558649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a674b8-c81a-4579-a439-41663c1af283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:37.558931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a674b8-c81a-4579-a439-41663c1af283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:37.570126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a674b8-c81a-4579-a439-41663c1af283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:37.571604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a674b8-c81a-4579-a439-41663c1af283 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:37.582979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 769/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:41.642458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3eb0-568a-4ff9-8482-8bf617297ab8 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:41.643021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3eb0-568a-4ff9-8482-8bf617297ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:41.643021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3eb0-568a-4ff9-8482-8bf617297ab8 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:41.644476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:41.651390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baa4fa-8464-4d96-806d-74b3d5255a4c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:41.652267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baa4fa-8464-4d96-806d-74b3d5255a4c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:41.652267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baa4fa-8464-4d96-806d-74b3d5255a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:41.671431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-33baa4fa-8464-4d96-806d-74b3d5255a4c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:41.672707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-33baa4fa-8464-4d96-806d-74b3d5255a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:41.681939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:41.982760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3eb0-568a-4ff9-8482-8bf617297ab8 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:41.984697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-421c3eb0-568a-4ff9-8482-8bf617297ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:42.032203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:42.060426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3eb0-568a-4ff9-8482-8bf617297ab8 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:42.066057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38addb41-53e6-49d8-93f4-b2383bd7dfbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:42.066984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38addb41-53e6-49d8-93f4-b2383bd7dfbc service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:42.067485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38addb41-53e6-49d8-93f4-b2383bd7dfbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:42.067977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38addb41-53e6-49d8-93f4-b2383bd7dfbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:42.083397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38addb41-53e6-49d8-93f4-b2383bd7dfbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:42.085778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38addb41-53e6-49d8-93f4-b2383bd7dfbc service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:42.098571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 766/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:44.587552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2da94-e911-4210-98f5-71bd3b2f07c8 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:44.588199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2da94-e911-4210-98f5-71bd3b2f07c8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:44.588199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2da94-e911-4210-98f5-71bd3b2f07c8 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:44.589556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:47:44.596336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc7c7-8ba6-4ec2-a4eb-3e1b6c2f81d8 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:44.597061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc7c7-8ba6-4ec2-a4eb-3e1b6c2f81d8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:44.597201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccc7c7-8ba6-4ec2-a4eb-3e1b6c2f81d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:44.625194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-67ccc7c7-8ba6-4ec2-a4eb-3e1b6c2f81d8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:44.626821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-67ccc7c7-8ba6-4ec2-a4eb-3e1b6c2f81d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:44.636082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:44.931264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2da94-e911-4210-98f5-71bd3b2f07c8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:44.933139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-54a2da94-e911-4210-98f5-71bd3b2f07c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:44.941184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=84 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:44.948060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2da94-e911-4210-98f5-71bd3b2f07c8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:44.951838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d3fc4-20b2-430d-b14c-6e902373cfcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:44.952530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d3fc4-20b2-430d-b14c-6e902373cfcb service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:44.952910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d3fc4-20b2-430d-b14c-6e902373cfcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:44.953104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d3fc4-20b2-430d-b14c-6e902373cfcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:44.964444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d3fc4-20b2-430d-b14c-6e902373cfcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:44.969086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d3fc4-20b2-430d-b14c-6e902373cfcb service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:44.977992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 772/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:54.618633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=13 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:54.988013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:16.123775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.124877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.125451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.157362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.158900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.170811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 768/1540] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.182753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.183435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.183536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.211704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.213066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.227366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 773/1541] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.233616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.248467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5913aa-42f0-4adf-8307-739ca007e988 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:16.257300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:16.258780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.259319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.259915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.260960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:16.261518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:16.262808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:49:16.262808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:49:16.262808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:49:16.264223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:16.284046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 769/1542] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:16.292557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.304187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd71e0-8b20-4438-ae3d-2af8a32978d6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:16.318642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:16.319345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.319633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.319916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.320690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:16.321963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:49:16.321963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:49:16.321963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:49:16.323224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522630f-2bd4-4e7c-b091-5d8a0ed3c455 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:16.442751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 774/1543] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:16.447252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.465581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80929282-8cc8-4948-ba50-d3ac271964af admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:16.557161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:16.557854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.558141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.558422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.559119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:16.560202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:49:16.560202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:49:16.560202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:49:16.562007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4c0c6-8ed5-4bc6-aadd-7079af5924a9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:16.589421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8124c2-9233-4d9a-8411-5b12483b12f8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.591414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8124c2-9233-4d9a-8411-5b12483b12f8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.591414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8124c2-9233-4d9a-8411-5b12483b12f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.618083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-cd8124c2-9233-4d9a-8411-5b12483b12f8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.625659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-cd8124c2-9233-4d9a-8411-5b12483b12f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.632890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 775/1544] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.634507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e126245-cd4c-4d40-8769-47a107d38ec7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.634811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e126245-cd4c-4d40-8769-47a107d38ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.635092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e126245-cd4c-4d40-8769-47a107d38ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.660674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4e126245-cd4c-4d40-8769-47a107d38ec7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.661949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4e126245-cd4c-4d40-8769-47a107d38ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.671330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 776/1545] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.672896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07294e-e249-493b-9284-7f228ee61d27 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.673195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07294e-e249-493b-9284-7f228ee61d27 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.673748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07294e-e249-493b-9284-7f228ee61d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.695389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7c07294e-e249-493b-9284-7f228ee61d27 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.696873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7c07294e-e249-493b-9284-7f228ee61d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.699840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 770/1546] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:16.701260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58350-1f21-4dda-b3ca-497623f02dde None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.705491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58350-1f21-4dda-b3ca-497623f02dde None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.705902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58350-1f21-4dda-b3ca-497623f02dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.706872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 777/1547] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.708391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b97f2-2de4-4847-ab73-d675c632bdaf None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.708732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b97f2-2de4-4847-ab73-d675c632bdaf None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.709069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b97f2-2de4-4847-ab73-d675c632bdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.733029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-8ac58350-1f21-4dda-b3ca-497623f02dde None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.734390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8ac58350-1f21-4dda-b3ca-497623f02dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.743400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 771/1548] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.744924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64f0d-6dd3-4d7b-82ef-a7af6dc16541 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.745222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64f0d-6dd3-4d7b-82ef-a7af6dc16541 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.745499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64f0d-6dd3-4d7b-82ef-a7af6dc16541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.772689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d5c64f0d-6dd3-4d7b-82ef-a7af6dc16541 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.773936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d5c64f0d-6dd3-4d7b-82ef-a7af6dc16541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.783091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 772/1549] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.784637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d4da8-7212-4557-92c2-fb2038c38e20 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.784933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d4da8-7212-4557-92c2-fb2038c38e20 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.785217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d4da8-7212-4557-92c2-fb2038c38e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.805993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a34d4da8-7212-4557-92c2-fb2038c38e20 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.807250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a34d4da8-7212-4557-92c2-fb2038c38e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.815707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 773/1550] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.817090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.817386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.817668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.837322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.838545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.847764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 774/1551] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.848379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.854769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-9d1b97f2-2de4-4847-ab73-d675c632bdaf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.855990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9d1b97f2-2de4-4847-ab73-d675c632bdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.862198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fb693-cc53-4653-a373-d54a4a3976af None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:16.868086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 778/1552] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.868086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3c481-d98e-40dc-8654-6c697af63e93 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.868086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3c481-d98e-40dc-8654-6c697af63e93 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.868086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3c481-d98e-40dc-8654-6c697af63e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.875468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc971f7-f60e-4b63-8ab9-2c5921584e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:16.877549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc971f7-f60e-4b63-8ab9-2c5921584e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.877549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc971f7-f60e-4b63-8ab9-2c5921584e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.877549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc971f7-f60e-4b63-8ab9-2c5921584e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.877549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc971f7-f60e-4b63-8ab9-2c5921584e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:16.879052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc971f7-f60e-4b63-8ab9-2c5921584e3c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:16.886044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-55a3c481-d98e-40dc-8654-6c697af63e93 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.887255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-55a3c481-d98e-40dc-8654-6c697af63e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.890979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 775/1553] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:16.892068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cc225-54df-4282-a769-690f7a1c459f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.892365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cc225-54df-4282-a769-690f7a1c459f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.892712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cc225-54df-4282-a769-690f7a1c459f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.897744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 779/1554] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.906702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.906702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.906702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.912004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4f2cc225-54df-4282-a769-690f7a1c459f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.917747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4f2cc225-54df-4282-a769-690f7a1c459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.919172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.920477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.921472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 776/1555] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.926272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a409d2a-9cbd-42bf-8b71-4f5a1392c5ae None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.926272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a409d2a-9cbd-42bf-8b71-4f5a1392c5ae None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.926272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a409d2a-9cbd-42bf-8b71-4f5a1392c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.930319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 780/1556] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.931112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.941230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e307bbc-25c0-45b2-875e-f2fb6fd63c2b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:16.946355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:16.946481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-8a409d2a-9cbd-42bf-8b71-4f5a1392c5ae None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.946584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.946793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.947126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.947458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8a409d2a-9cbd-42bf-8b71-4f5a1392c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.947860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:16.948192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:16.949886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:16.955883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 777/1557] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.959532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.959532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.959532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.973471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 781/1558] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:16.973471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.976866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:16.980705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd8899-34a1-4239-aaa5-fc3ac7342ee8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:16.980833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:16.982477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:16.983374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:16.983667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:16.983935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:16.984693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:16.985001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:16.986278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 778/1559] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:16.986587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e1949-05eb-4871-8f24-5b9a611d97f9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:16.987048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:16.996470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be15b9-5cab-4ca0-82c2-db34749fbe8c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.003216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 782/1560] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:17.003388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.003388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.003388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.003648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.003745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.003827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.003827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.004368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.005308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.017112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 779/1561] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:17.017991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.022208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:17.023411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:17.027331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa5e66-9b65-410d-a255-52890a2ec527 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.031745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 783/1562] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:17.032002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.036550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.036668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.036668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.036668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.036668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:17.036668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.036668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.043638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa772a-0e08-4af8-be9b-100820f7b378 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.048817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.053877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.054028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 780/1563] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:17.054131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.054131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.054298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.054380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:17.054380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.054380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.063748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7a28a-373a-4716-82dc-226dc8275b6c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.068281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.068729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 784/1564] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:17.068961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.069242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.069445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.069618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.070298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:17.070605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.072201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.092450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 781/1565] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:17.092450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.092782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bd758-6a8c-4ea8-9e57-333e06c83087 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.092782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.092782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.092782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.093098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.093448 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.095367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.095941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e55af3-bc49-411a-9ee6-397e4c30fae2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.100941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.101618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.101904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.102180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.102882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:17.103186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.104833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.116884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 782/1566] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:17.127673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.127673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf6afa-70bc-454d-beb3-51ab0c7795c7 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.133588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.134250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.134529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.134800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.135476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.137394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.195242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 785/1567] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.195242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.243475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:17.253056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 783/1568] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.253967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.259567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7188e94-5269-4ae3-83f4-3c8613ce1a4d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.289941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:17.317671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80fde0-e58e-4ff1-b0a4-b902e37da4f9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.318590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.319794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.319941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.320115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] PATH_INFO: `/v3/projects/a95b9c4a13b7410389f01481d7446b8b/users/4f5e5354889c469c94a217d25b684c4d/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.322304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.328444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.328444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.328444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.328444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:17.328444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.328444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.336774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a95b9c4a13b7410389f01481d7446b8b, user_id=4f5e5354889c469c94a217d25b684c4d, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.337835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:49:17.337835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:49:17.337835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:49:17.338965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 784/1569] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:17.339823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.339952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.349427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923f87-86a0-4f77-9d50-1627810eff1f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.353974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.354615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.354897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.355173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.355853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.358292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.428405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 786/1570] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:17 2022] PUT /identity/v3/projects/a95b9c4a13b7410389f01481d7446b8b/users/4f5e5354889c469c94a217d25b684c4d/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:17.429340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.455286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 785/1571] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.455286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.503551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb501de3-7e44-4352-957c-4c5017b3a48b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.521007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bed368-7f8b-4a6d-8621-5c149c07866a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.526886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.531644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.531644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.531644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.531644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.531644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.552041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.552764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.553047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.553327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.554018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.555955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.608522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 786/1572] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.609341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.637937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 787/1573] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.637937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.692174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:17.707564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33862b9-6ae3-45bc-bb27-96428f73f26e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.712803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.713529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.713786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.714232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.714940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.716997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.800190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:17.804029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1b72-50e0-4441-ad45-e9b81fcafa0f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.808763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.809430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.809715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.809993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.810855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.813211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.824077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 788/1574] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.827851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.834821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960142aa-a057-41ca-9e47-c7b028838956 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:17.839611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255dd54-2b14-45ad-9a33-9071fd819ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:17.840265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255dd54-2b14-45ad-9a33-9071fd819ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.840575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255dd54-2b14-45ad-9a33-9071fd819ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.840857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255dd54-2b14-45ad-9a33-9071fd819ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.841533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255dd54-2b14-45ad-9a33-9071fd819ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:17.844133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255dd54-2b14-45ad-9a33-9071fd819ad9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:17.939977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 787/1575] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 383 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.940873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:17.963051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 789/1576] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:17.963051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:17.968813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:17.968813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:17.995205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] MFA Rules not processed for user `4f5e5354889c469c94a217d25b684c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:17.996441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:18.005705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5808-4021-469d-b640-6fe0c7f67d25 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.010369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.011149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.011437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.011719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.012412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.015076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.062577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 790/1577] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:18.063486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.074130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c0f83-4436-4273-acc3-8fce612852f8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.080178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.081032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.081111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.081453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.082227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.084151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.102242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 788/1578] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:17 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:18.103131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.166865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b6ece-318e-4221-a9e3-5038ff0b073c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.171929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.172656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.172944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.173244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.173932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.175926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.220578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 791/1579] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:18 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:18.221536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.232247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523c0c5-8597-406d-bee6-c95a37c594c0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.237118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.237781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.238065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.238342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.239056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.241006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.301424 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 789/1580] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:18 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:18.302463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.312547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b7ce-8293-416c-b12a-9febe5b525b7 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.317147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.317742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.318070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.318349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.319061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.320718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.331718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 790/1581] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.332632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.336331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 792/1582] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:18 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:18.337227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.343784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b247e8-70f7-4c45-aca2-a07333c8c64d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.348740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.349411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.349692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.349967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.350649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.351250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649f054-b766-4c3d-bc10-64e9e7d28602 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.352755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.356189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d3eb-4f12-4730-9fd6-32788ee3d296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.356891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d3eb-4f12-4730-9fd6-32788ee3d296 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.357180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d3eb-4f12-4730-9fd6-32788ee3d296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.357473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d3eb-4f12-4730-9fd6-32788ee3d296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.358198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6d3eb-4f12-4730-9fd6-32788ee3d296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.359782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6d3eb-4f12-4730-9fd6-32788ee3d296 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.371188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 793/1583] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.372391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074001-73e1-466c-8d10-491d2a0308fc None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.372707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074001-73e1-466c-8d10-491d2a0308fc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.372985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074001-73e1-466c-8d10-491d2a0308fc None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.374813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 794/1584] 10.222.0.58 () {54 vars in 961 bytes} [Fri May 27 03:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:18.375536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074001-73e1-466c-8d10-491d2a0308fc None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.388871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074001-73e1-466c-8d10-491d2a0308fc None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.394598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.395249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.395655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.395944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.396654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.398568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.447557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 791/1585] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:18 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:18.448396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.458556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edc9c2-71d8-462c-be22-93c40fc16382 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.463358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.464025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.464310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.464620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.465377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.467035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.478394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 792/1586] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.479307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.489587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed5a12-814f-4d34-b00b-b02557baff46 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.493225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 795/1587] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:18 2022] POST /identity/v3/users => generated 282 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:18.494199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.495075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.504614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.504614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.504614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.504614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.504614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.505414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cbbd3-142f-4cf0-8ea3-25bccdd8854f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.510334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.510540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 793/1588] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.511367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.511481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.511652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.512122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.512707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.515151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.523993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8c7e5-7fac-40a8-bd82-63052fd41a55 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.525848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 796/1589] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.526805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.532846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d6d8a-d77b-451c-a48b-196d855065ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.533537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d6d8a-d77b-451c-a48b-196d855065ce admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.533885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d6d8a-d77b-451c-a48b-196d855065ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.534279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d6d8a-d77b-451c-a48b-196d855065ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.535017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d6d8a-d77b-451c-a48b-196d855065ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.536925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d6d8a-d77b-451c-a48b-196d855065ce admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.537688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256b2be-5b77-498e-86aa-2ada74a6ec1f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.538428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:18.542538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.543275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.543560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.543839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.544592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.546442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.548197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 794/1590] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.549563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.562853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.562853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.563609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 797/1591] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.563609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.571785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:18.572006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523939c-8b61-47ba-a211-c38ddac6f664 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.572131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:18.576041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.578004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.578315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.578602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.579339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.581002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.591706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 798/1592] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.593168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.603020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96f21-1067-440b-b4cc-34d02cf1a338 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.608904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.609568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.609855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.610139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.610865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.612860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.625397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=85,keystone:INSERT=15 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:18.630480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 799/1593] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.631459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.644930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9d717-027e-4c20-99cf-457ef7b94d4d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.648092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 795/1594] 10.222.0.58 () {58 vars in 1058 bytes} [Fri May 27 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:18.649356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.650569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.651498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.651888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.652200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.653049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.654842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.665186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e068832-4bb9-41ad-81aa-a022ecbae738 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.671012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 800/1595] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.671012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.673244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.673930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.674218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.674492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.675372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.677118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.682024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab140e4e-a4c4-4299-8faa-a9392f93cb8c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.687788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.688627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 796/1596] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.688797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.688996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.689266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.689569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] PATH_INFO: `/v3/projects/b1f668cbb60844d881dfb2af61132af7/users/bca16fe930244e12bbd12a23323d583e/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.701224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e3ecf-872e-43bd-820c-586a73820c46 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.706798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f668cbb60844d881dfb2af61132af7, user_id=bca16fe930244e12bbd12a23323d583e, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.707748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.708421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.708802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.709155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.709605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.709900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.712720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.723890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 797/1597] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.726037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.735246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a2fa5-f1bd-4c95-9f33-3800612f5922 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.740055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.741016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.741304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.741585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] PATH_INFO: `/v3/projects/df0a2a8dca064a3ea1c20b7a32842000/users/65c434b614e34b1f9b79375d85720e72/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.757562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0a2a8dca064a3ea1c20b7a32842000, user_id=65c434b614e34b1f9b79375d85720e72, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.760313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.805704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 801/1598] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:18 2022] PUT /identity/v3/projects/b1f668cbb60844d881dfb2af61132af7/users/bca16fe930244e12bbd12a23323d583e/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:18.806533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.836592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 798/1599] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:18 2022] PUT /identity/v3/projects/df0a2a8dca064a3ea1c20b7a32842000/users/65c434b614e34b1f9b79375d85720e72/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:18.841040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:18.868995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5923ad5-8fa0-4e78-ae84-d7abcc5f1c08 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.907848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef393a3-f948-4d24-bda3-0b8910c06292 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:18.912801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.913697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.913961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.914230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] PATH_INFO: `/v3/projects/7e34d34b28b44d399fe2a1e011771f22/users/0ebc2cec00384f28bb79d0742d3bccf1/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.928065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:18.928773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:18.929070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:18.929315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e34d34b28b44d399fe2a1e011771f22, user_id=0ebc2cec00384f28bb79d0742d3bccf1, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.929441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:18.930062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:18.931651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.931954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:18.941886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 802/1600] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:18.942710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.023107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b055b4-f8bb-42af-9fa0-cbc721b187dd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.029630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.032550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.032550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.032550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.058796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 799/1601] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:18 2022] PUT /identity/v3/projects/7e34d34b28b44d399fe2a1e011771f22/users/0ebc2cec00384f28bb79d0742d3bccf1/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:19.059607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.094823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:19.095031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.095630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.104760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 803/1602] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:19.105550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.116369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba7d07-5119-406b-a29b-434ff2878814 service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.121048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0b40d-9f73-47b2-b95c-564ed81db5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.121984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0b40d-9f73-47b2-b95c-564ed81db5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.122264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0b40d-9f73-47b2-b95c-564ed81db5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.122611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0b40d-9f73-47b2-b95c-564ed81db5e9 admin admin] PATH_INFO: `/v3/projects/26b40f0d69f44da79dfb780da80d4893/users/3cfb66d6bc254e549d6d355c2bbb440a/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.126387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd8fb7-9bfd-485e-9500-399b2db63a0b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.131647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.132405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.132789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.133144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.134036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.135662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.139718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0b40d-9f73-47b2-b95c-564ed81db5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b40f0d69f44da79dfb780da80d4893, user_id=3cfb66d6bc254e549d6d355c2bbb440a, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.142507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0b40d-9f73-47b2-b95c-564ed81db5e9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.146791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 800/1603] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:19.147587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.161372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:19.163691 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0a75b-c1dd-4e69-bf2f-fa63ae68fb18 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.169429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472a57-9485-4c1c-83b0-1acebeac5f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.171052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472a57-9485-4c1c-83b0-1acebeac5f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.171337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472a57-9485-4c1c-83b0-1acebeac5f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.171614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472a57-9485-4c1c-83b0-1acebeac5f2f admin admin] PATH_INFO: `/v3/projects/4879215a268c47c782a6400a394b3968/users/df05bff695ce4889bc597c9437142799/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.187291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472a57-9485-4c1c-83b0-1acebeac5f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4879215a268c47c782a6400a394b3968, user_id=df05bff695ce4889bc597c9437142799, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.190017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472a57-9485-4c1c-83b0-1acebeac5f2f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.235471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 804/1604] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:19 2022] PUT /identity/v3/projects/26b40f0d69f44da79dfb780da80d4893/users/3cfb66d6bc254e549d6d355c2bbb440a/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:19.236795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.237090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.237489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.254930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.256153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.288869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 801/1605] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:19 2022] PUT /identity/v3/projects/4879215a268c47c782a6400a394b3968/users/df05bff695ce4889bc597c9437142799/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:19.291286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36333bb-e7d9-4ea7-a552-ad413a5729ba None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.291645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36333bb-e7d9-4ea7-a552-ad413a5729ba None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.291999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36333bb-e7d9-4ea7-a552-ad413a5729ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.332132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a36333bb-e7d9-4ea7-a552-ad413a5729ba None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.333430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a36333bb-e7d9-4ea7-a552-ad413a5729ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.360320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 805/1606] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.361251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.376836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b7f1c-3bd8-45aa-824e-000bcfd72fb9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.409713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 802/1607] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.410969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.411266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.411541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.429787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.431107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.442475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 803/1608] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.442475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.449870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48a93-0cec-4f46-a5d1-db89bdcb32f2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.455207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c27b1a-566a-45af-8d15-3fc5fbb77fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.456090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27b1a-566a-45af-8d15-3fc5fbb77fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.456373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27b1a-566a-45af-8d15-3fc5fbb77fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.456611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8071324-42eb-4775-9f5f-3bddb3cff9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.456836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c27b1a-566a-45af-8d15-3fc5fbb77fd7 admin admin] PATH_INFO: `/v3/projects/024ccdd4520b48ef823b628e2a2ec915/users/4860ece0452c42aca17c0cda0d673ca9/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.458054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8071324-42eb-4775-9f5f-3bddb3cff9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.458349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8071324-42eb-4775-9f5f-3bddb3cff9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.458631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8071324-42eb-4775-9f5f-3bddb3cff9c6 admin admin] PATH_INFO: `/v3/projects/967f4d957acb42e29d0e065ad5f349e8/users/b522896d75aa494cb371bd92ee77064a/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.475070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8071324-42eb-4775-9f5f-3bddb3cff9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967f4d957acb42e29d0e065ad5f349e8, user_id=b522896d75aa494cb371bd92ee77064a, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.475271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27b1a-566a-45af-8d15-3fc5fbb77fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=024ccdd4520b48ef823b628e2a2ec915, user_id=4860ece0452c42aca17c0cda0d673ca9, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.478474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8071324-42eb-4775-9f5f-3bddb3cff9c6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.478725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c27b1a-566a-45af-8d15-3fc5fbb77fd7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.567479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 806/1609] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:19 2022] PUT /identity/v3/projects/967f4d957acb42e29d0e065ad5f349e8/users/b522896d75aa494cb371bd92ee77064a/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:19.568488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.568813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.569097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.589735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:19.597792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 804/1610] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:19 2022] PUT /identity/v3/projects/024ccdd4520b48ef823b628e2a2ec915/users/4860ece0452c42aca17c0cda0d673ca9/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:19.599589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.599816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.600183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.600298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.601534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.634694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] MFA Rules not processed for user `df05bff695ce4889bc597c9437142799`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.636043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.671867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 807/1611] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.672585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.697155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:19.706496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 805/1612] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.707440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.730198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e1ada-6edc-41b3-b2da-c894ef45a4b7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.734835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.735498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.735784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.736063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.745579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.747021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.757577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 808/1613] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:19.758656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.760009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.766990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338e169-8ab2-49a0-8564-51aa0097ae22 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.776897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcce696-83d8-4390-9c2c-ee0584c21e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.776897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcce696-83d8-4390-9c2c-ee0584c21e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.777132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f79d94-291f-4f11-832d-6d82301a1a1f service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.777226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcce696-83d8-4390-9c2c-ee0584c21e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.777226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcce696-83d8-4390-9c2c-ee0584c21e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.777364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162afb32-0bcf-47ab-82fe-de141bf35b97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.777364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162afb32-0bcf-47ab-82fe-de141bf35b97 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.777364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162afb32-0bcf-47ab-82fe-de141bf35b97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.777364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162afb32-0bcf-47ab-82fe-de141bf35b97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.782832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcce696-83d8-4390-9c2c-ee0584c21e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.784366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcce696-83d8-4390-9c2c-ee0584c21e12 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.787329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162afb32-0bcf-47ab-82fe-de141bf35b97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.788941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162afb32-0bcf-47ab-82fe-de141bf35b97 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.794160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 806/1614] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:19.799809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f5f6-d5fd-4f09-ace6-650395a449c4 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.799809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f5f6-d5fd-4f09-ace6-650395a449c4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.799809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8f5f6-d5fd-4f09-ace6-650395a449c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.800384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 809/1615] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:19.801789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.802022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.802305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.827686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c9d8f5f6-d5fd-4f09-ace6-650395a449c4 None None] MFA Rules not processed for user `b522896d75aa494cb371bd92ee77064a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.829487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c9d8f5f6-d5fd-4f09-ace6-650395a449c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.837107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] MFA Rules not processed for user `4860ece0452c42aca17c0cda0d673ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:19.838447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:19.915016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 807/1616] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.915245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 810/1617] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:19.915245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.916935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24c778-b1bc-41b0-870f-fcd41d117d44 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.917990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24c778-b1bc-41b0-870f-fcd41d117d44 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.918704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24c778-b1bc-41b0-870f-fcd41d117d44 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.921479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 808/1618] 10.222.0.58 () {54 vars in 961 bytes} [Fri May 27 03:49:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:19.922334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24c778-b1bc-41b0-870f-fcd41d117d44 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.931916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6513a2-d34b-48f1-b88b-fd3fbe180771 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.936550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.937697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.938003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.938334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.951120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.955848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:19.970878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 811/1619] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:19.971867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:19.982928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d2544-cdc5-43b7-b3d2-33aa8bbd4513 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:19.988533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50d3aa-d37d-4511-b74e-d8b6d741a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:19.989437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50d3aa-d37d-4511-b74e-d8b6d741a2df admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:19.989741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50d3aa-d37d-4511-b74e-d8b6d741a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:19.990120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50d3aa-d37d-4511-b74e-d8b6d741a2df admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:19.990882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50d3aa-d37d-4511-b74e-d8b6d741a2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:19.993214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50d3aa-d37d-4511-b74e-d8b6d741a2df admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.005539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:20.008201 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24c778-b1bc-41b0-870f-fcd41d117d44 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.013389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.015853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.015990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.016206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.026627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.028141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.039146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 809/1620] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:20.039984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.051720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71bb4-0e43-477d-8431-ad6405bc5811 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.056602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.057258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.057549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.057830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.058525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.060493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.123968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 812/1621] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:19 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:20.125447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.125810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.126166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.147821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:20.150299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:20.197592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 810/1622] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:20 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:20.198047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.224494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:20.268254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 813/1623] 10.222.0.58 () {58 vars in 1058 bytes} [Fri May 27 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:20.270659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.333237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ffd3c-3b4a-4419-ab9c-28a207c2fdd3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.344349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb00a9-cdc2-4179-8499-b2b5a28784e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.346512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb00a9-cdc2-4179-8499-b2b5a28784e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.347182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb00a9-cdc2-4179-8499-b2b5a28784e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.347508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb00a9-cdc2-4179-8499-b2b5a28784e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.348258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb00a9-cdc2-4179-8499-b2b5a28784e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.350437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb00a9-cdc2-4179-8499-b2b5a28784e7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.390008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede83a46-0c5f-4f7c-a9bc-a23bbe86f776 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.396285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55516b61-4788-4a93-859f-bcee2b3bee35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.397194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55516b61-4788-4a93-859f-bcee2b3bee35 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.397664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55516b61-4788-4a93-859f-bcee2b3bee35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.398003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55516b61-4788-4a93-859f-bcee2b3bee35 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.398804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55516b61-4788-4a93-859f-bcee2b3bee35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.401543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55516b61-4788-4a93-859f-bcee2b3bee35 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.485211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 811/1624] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:20 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:20.488845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb51c8d-a5b5-45e2-8160-40469d2d1965 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.489807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb51c8d-a5b5-45e2-8160-40469d2d1965 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.490299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb51c8d-a5b5-45e2-8160-40469d2d1965 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.494751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 812/1625] 10.222.0.58 () {54 vars in 912 bytes} [Fri May 27 03:49:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:20.495625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb51c8d-a5b5-45e2-8160-40469d2d1965 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.516710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb51c8d-a5b5-45e2-8160-40469d2d1965 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.521426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 814/1626] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:20 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:20.523085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.524076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.524618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.525099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.526191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.528760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.529166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.529442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.529776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.543440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 813/1627] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:20.544740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.575588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a2407-27c4-4959-98b6-f780e11f2061 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.578921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:20.581544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:20.589796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.590856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.591492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.592156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.593163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.595423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.636903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 814/1628] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:20.637790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.664016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3d4a8-0b3a-4956-af79-e33a9bac3000 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.673094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.674306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.674828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.675307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] PATH_INFO: `/v3/projects/13f8766f95b04e2e8ddd09600184e85a/users/6e84ba980b334c5086c496fad707d597/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.692047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f8766f95b04e2e8ddd09600184e85a, user_id=6e84ba980b334c5086c496fad707d597, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.695139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.710392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 815/1629] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:20.711559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.716862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:20.724732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab17bd-b1b5-423f-8a26-bc1daa3369e4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.731291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.732467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.733536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.734166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] PATH_INFO: `/v3/projects/e2b2d69babfc451ea96be5c9938ce90e/users/508d7d006b7d46a78d0d9e534ddaefff/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:20.753504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b2d69babfc451ea96be5c9938ce90e, user_id=508d7d006b7d46a78d0d9e534ddaefff, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:20.757486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:20.784531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 815/1630] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:20 2022] PUT /identity/v3/projects/13f8766f95b04e2e8ddd09600184e85a/users/6e84ba980b334c5086c496fad707d597/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:20.785678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.837109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 816/1631] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:20 2022] PUT /identity/v3/projects/e2b2d69babfc451ea96be5c9938ce90e/users/508d7d006b7d46a78d0d9e534ddaefff/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:20.837980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:20.876608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f3cc9-77f8-4551-8401-601fc9b30fe5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.991405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936875b2-c5a6-4cf3-a34b-0b304afc33bf admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:20.995660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18fcaa-9cac-4786-a876-b06ac6fa1803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:20.997709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18fcaa-9cac-4786-a876-b06ac6fa1803 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:20.999282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18fcaa-9cac-4786-a876-b06ac6fa1803 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:20.999950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18fcaa-9cac-4786-a876-b06ac6fa1803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.004250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.012312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.012312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.012312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.012312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.012312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.023350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 817/1632] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:21.024269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.036729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564d700-bfe7-404d-a91c-0f8f73c8cafc admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.043638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:21.045016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.046457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.046743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.047015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.048489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.050294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.066263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 818/1633] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:21.066263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.080445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fffb48-f446-48ed-b3ba-4df366ed4dd8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.090369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.091701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.092071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.092527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] PATH_INFO: `/v3/projects/13f8766f95b04e2e8ddd09600184e85a/users/6e84ba980b334c5086c496fad707d597/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.096073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f8766f95b04e2e8ddd09600184e85a, user_id=6e84ba980b334c5086c496fad707d597, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.099136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.113262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18fcaa-9cac-4786-a876-b06ac6fa1803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.115414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18fcaa-9cac-4786-a876-b06ac6fa1803 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.124966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 819/1634] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:21 2022] PUT /identity/v3/projects/13f8766f95b04e2e8ddd09600184e85a/users/6e84ba980b334c5086c496fad707d597/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:21.127061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.137374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 816/1635] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:21.139023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.139316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.140050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.200912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] MFA Rules not processed for user `6e84ba980b334c5086c496fad707d597`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:21.203279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:21.281155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5a746-a68d-4bd7-9f75-1d427557a85f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.285357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8cfe8-23c9-47a1-8016-8972b72c78ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.286119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cfe8-23c9-47a1-8016-8972b72c78ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.286654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cfe8-23c9-47a1-8016-8972b72c78ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.286654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cfe8-23c9-47a1-8016-8972b72c78ff admin admin] PATH_INFO: `/v3/projects/e2b2d69babfc451ea96be5c9938ce90e/users/508d7d006b7d46a78d0d9e534ddaefff/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.288770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8cfe8-23c9-47a1-8016-8972b72c78ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b2d69babfc451ea96be5c9938ce90e, user_id=508d7d006b7d46a78d0d9e534ddaefff, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.294664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 817/1636] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:21.294857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8cfe8-23c9-47a1-8016-8972b72c78ff admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.311451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.324261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 820/1637] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:21 2022] PUT /identity/v3/projects/e2b2d69babfc451ea96be5c9938ce90e/users/508d7d006b7d46a78d0d9e534ddaefff/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:21.336803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e21f28-1813-4a57-a7d5-2c2f6a7b9452 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.338689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e21f28-1813-4a57-a7d5-2c2f6a7b9452 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.338689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e21f28-1813-4a57-a7d5-2c2f6a7b9452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.374641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b6e21f28-1813-4a57-a7d5-2c2f6a7b9452 None None] MFA Rules not processed for user `508d7d006b7d46a78d0d9e534ddaefff`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:21.375956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b6e21f28-1813-4a57-a7d5-2c2f6a7b9452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:21.387545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf308e-190e-4954-ba09-a2b8b7652dfa None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.423997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:21.475806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.476671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.477005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.477282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.504943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 821/1638] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:21.504943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82199c2d-f7db-4edc-b1c2-4a6fe8c819c1 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.504943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82199c2d-f7db-4edc-b1c2-4a6fe8c819c1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.505174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82199c2d-f7db-4edc-b1c2-4a6fe8c819c1 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.507377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 822/1639] 10.222.0.58 () {54 vars in 912 bytes} [Fri May 27 03:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:21.513860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.514164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.514436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.527973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:21.546716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:21.547958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:21.597487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.598969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.863321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 818/1640] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:21.863321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.863321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09663d3-3ed0-498b-ba73-76c8d044df0f service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.863321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.867348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.867348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.867348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.872824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 823/1641] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:21.872824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.877404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.877404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.877404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 819/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:21.877404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.877404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1649ce-11ff-46c1-afb8-cf60d0fd4129 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.906292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.906292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.906292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.906292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.909515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.909515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.911501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5603df-e874-46d2-b83c-ffd0d5a1a5a7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.911501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.911501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.911501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.913006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] PATH_INFO: `/v3/users/b522896d75aa494cb371bd92ee77064a` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.913006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b522896d75aa494cb371bd92ee77064a)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.913006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: WARNING py.warnings [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:49:21.913006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: warnings.warn(msg) May 27 03:49:21.913006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: May 27 03:49:21.913006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:21.915673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 820/1643] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:21.915673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:21.915673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87ff5-dde4-40e1-8ee9-74b65b47b07e service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:21.915673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:21.920400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:21.920400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:21.920400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] PATH_INFO: `/v3/users/4f5e5354889c469c94a217d25b684c4d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:21.920400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f5e5354889c469c94a217d25b684c4d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:21.920400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:22.117763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b522896d75aa494cb371bd92ee77064a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.118585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b522896d75aa494cb371bd92ee77064a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.119310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b522896d75aa494cb371bd92ee77064a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.144356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b522896d75aa494cb371bd92ee77064a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.161208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-fe59394e-7975-4a6a-9bb7-224b78eabe49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b522896d75aa494cb371bd92ee77064a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.223972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f5e5354889c469c94a217d25b684c4d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.224801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f5e5354889c469c94a217d25b684c4d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.234233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f5e5354889c469c94a217d25b684c4d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.234989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f5e5354889c469c94a217d25b684c4d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.303512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 824/1644] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:21 2022] DELETE /identity/v3/users/b522896d75aa494cb371bd92ee77064a => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:22.303512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5af5e-34c8-480d-a4ec-5a0d33a5126c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.303512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5af5e-34c8-480d-a4ec-5a0d33a5126c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.304154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5af5e-34c8-480d-a4ec-5a0d33a5126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.314573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:22.342129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-aed5af5e-34c8-480d-a4ec-5a0d33a5126c None None] MFA Rules not processed for user `0ebc2cec00384f28bb79d0742d3bccf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:22.343646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-aed5af5e-34c8-480d-a4ec-5a0d33a5126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.418245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3e016f99-97cf-4d00-8834-e5ae1ceddb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f5e5354889c469c94a217d25b684c4d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:22.431100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 825/1645] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:22.432783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9d60-309a-4d7e-97e5-fa2189369121 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.433077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9d60-309a-4d7e-97e5-fa2189369121 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.433358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9d60-309a-4d7e-97e5-fa2189369121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.453845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6e6f9d60-309a-4d7e-97e5-fa2189369121 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:22.455351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6e6f9d60-309a-4d7e-97e5-fa2189369121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.466138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 821/1646] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:21 2022] DELETE /identity/v3/users/4f5e5354889c469c94a217d25b684c4d => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:22.471865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.472161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.472452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.539335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 826/1647] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:22.541327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.541767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.542180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.553739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] MFA Rules not processed for user `0ebc2cec00384f28bb79d0742d3bccf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:22.554990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.563997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:22.565578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.627936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:22.649166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 827/1648] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:22.650328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:22.652074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.661200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 822/1649] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:22.674567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:22.676016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.747260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12ee4-e56f-460c-9052-45bea01744b0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:22.752209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:22.753139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.754443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.755136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.768027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:22.769890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:22.782060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 828/1650] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:22.782927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:22.785246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:22.805789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd55abc-e335-478b-ade5-2755a0e0ed5e service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:22.812355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:22.813149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.813438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.813758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.831883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:22.832235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6e95d-230a-4160-959a-84670ec7f718 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:22.834063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:22.844594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d223e-4565-46ea-9376-22c6c1b8ba9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:22.849618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d223e-4565-46ea-9376-22c6c1b8ba9b service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:22.849618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d223e-4565-46ea-9376-22c6c1b8ba9b service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:22.849618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d223e-4565-46ea-9376-22c6c1b8ba9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:22.861808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d223e-4565-46ea-9376-22c6c1b8ba9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:22.863452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d223e-4565-46ea-9376-22c6c1b8ba9b service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:22.869430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 829/1651] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:22.900311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 823/1652] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:49:23.288069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:23.291619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:23.316569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35f9c6-d433-44e2-bac6-9b357d75e0f1 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:23.325012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:23.326186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:23.327909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:23.328252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] PATH_INFO: `/v3/users/4860ece0452c42aca17c0cda0d673ca9` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:23.329843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4860ece0452c42aca17c0cda0d673ca9)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:23.332353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:23.539950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:23.663746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4860ece0452c42aca17c0cda0d673ca9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:23.663746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4860ece0452c42aca17c0cda0d673ca9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:23.664831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4860ece0452c42aca17c0cda0d673ca9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:23.696304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4860ece0452c42aca17c0cda0d673ca9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:23.710048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1bd7aa09-007a-4327-8383-b9fec6c48b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4860ece0452c42aca17c0cda0d673ca9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:23.746583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89ea3-8db4-4057-a5eb-f2c29b36fbab None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:23.747062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89ea3-8db4-4057-a5eb-f2c29b36fbab None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:23.747436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89ea3-8db4-4057-a5eb-f2c29b36fbab None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:23.756456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 824/1653] 10.222.0.58 () {54 vars in 912 bytes} [Fri May 27 03:49:23 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:23.762301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:23.762656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:23.762941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:23.833245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 830/1654] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:23 2022] DELETE /identity/v3/users/4860ece0452c42aca17c0cda0d673ca9 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:23.839381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:23.841250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:23.841368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:23.934135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:23.935362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:23.939799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:23.940810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.085711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 825/1655] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:49:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:24.087029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:24.088407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.147857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 831/1656] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:24.148885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:24.161404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.174350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ead035-91e1-4aa4-a0d6-d1201ad1e0a1 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:24.181758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:24.182657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:24.182947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:24.183598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] PATH_INFO: `/v3/projects/967f4d957acb42e29d0e065ad5f349e8` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:24.185047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967f4d957acb42e29d0e065ad5f349e8)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:24.187258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: WARNING py.warnings [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:49:24.187258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: warnings.warn(msg) May 27 03:49:24.187258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: May 27 03:49:24.188715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:24.196862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe171d8e-c2af-42d3-a841-6c6ff7c436d2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:24.204472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:24.205597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:24.205729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:24.206037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:24.216778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:24.220204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:24.231942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 832/1657] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:24.234074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:24.235605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.254483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b5cf1-2521-45f9-a678-b2657d5656a1 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:24.260449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:24.265499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:24.265940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:24.266337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] PATH_INFO: `/v3/projects/a95b9c4a13b7410389f01481d7446b8b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:24.268366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a95b9c4a13b7410389f01481d7446b8b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:24.270933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:24.368623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967f4d957acb42e29d0e065ad5f349e8', 'request_id': 'req-33ed2a1c-1493-415c-a831-99d39a068831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.387725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967f4d957acb42e29d0e065ad5f349e8', 'request_id': 'req-33ed2a1c-1493-415c-a831-99d39a068831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.488299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967f4d957acb42e29d0e065ad5f349e8', 'request_id': 'req-33ed2a1c-1493-415c-a831-99d39a068831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.493344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a95b9c4a13b7410389f01481d7446b8b', 'request_id': 'req-d62de369-142c-405f-89cd-18da1aee70d4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.537669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] The token cache is being invalidate because project 967f4d957acb42e29d0e065ad5f349e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:24.538513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.540055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.543030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 826/1658] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:24 2022] DELETE /identity/v3/projects/967f4d957acb42e29d0e065ad5f349e8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:24.544900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:24.546870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.607339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a95b9c4a13b7410389f01481d7446b8b', 'request_id': 'req-d62de369-142c-405f-89cd-18da1aee70d4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.621780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a95b9c4a13b7410389f01481d7446b8b', 'request_id': 'req-d62de369-142c-405f-89cd-18da1aee70d4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.658349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] The token cache is being invalidate because project a95b9c4a13b7410389f01481d7446b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:24.661145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.668578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:24.673627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 833/1659] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:24 2022] DELETE /identity/v3/projects/a95b9c4a13b7410389f01481d7446b8b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:24.674658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:24.680849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.699174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:24.750069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed2a1c-1493-415c-a831-99d39a068831 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:24.914642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:24.916242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:24.916720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:24.917178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:24.919863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:24.953192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de369-142c-405f-89cd-18da1aee70d4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:24.962090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:24.962280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:24.963712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6f4e5-c997-4504-ac5c-3730e10fe094 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:24.965216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:24.965699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:24.966469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] PATH_INFO: `/v3/users/6e84ba980b334c5086c496fad707d597` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:24.971356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e84ba980b334c5086c496fad707d597)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:24.976103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:24.986958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 827/1660] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:24.988303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd0724-6463-4041-8c96-fdd5c01bcedd None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:24.988786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd0724-6463-4041-8c96-fdd5c01bcedd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:24.989066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd0724-6463-4041-8c96-fdd5c01bcedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.065152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4bdd0724-6463-4041-8c96-fdd5c01bcedd None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:25.066493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4bdd0724-6463-4041-8c96-fdd5c01bcedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:25.128395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 828/1661] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:25.129911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.130202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.130479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.200812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:25.202563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:25.230317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 829/1662] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:25.234958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.249436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e84ba980b334c5086c496fad707d597'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:25.251321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e84ba980b334c5086c496fad707d597'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:25.252326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e84ba980b334c5086c496fad707d597'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:25.254855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c02ee-96d5-4f73-8743-9a8cceaef8ec None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.290724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.295195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.296463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.296993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.298018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:25.298562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.301202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1dc6de-c5ca-4409-aa79-86d5d55f52e2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:25.307139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e84ba980b334c5086c496fad707d597'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:25.322286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f8e1f9ee-73ad-4b33-96c9-9f6413ff5fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e84ba980b334c5086c496fad707d597'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:25.332615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 830/1663] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:49:25.334234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd29ce4-2080-48b0-8d45-8e23daf1816d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.334628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd29ce4-2080-48b0-8d45-8e23daf1816d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.335037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd29ce4-2080-48b0-8d45-8e23daf1816d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.337577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 831/1664] 10.222.0.58 () {54 vars in 960 bytes} [Fri May 27 03:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:25.339171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd29ce4-2080-48b0-8d45-8e23daf1816d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.347916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ffd29ce4-2080-48b0-8d45-8e23daf1816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:25.371257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd29ce4-2080-48b0-8d45-8e23daf1816d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.378392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.379242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.379909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.379909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.380830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.382872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:25.456366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 834/1665] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:24 2022] DELETE /identity/v3/users/6e84ba980b334c5086c496fad707d597 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:25.456366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.456366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.456366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.479673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:25.488341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:25.489650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:25.538638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 832/1666] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:25 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:25.539751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.574237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:25.625833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 835/1667] 10.222.0.58 () {58 vars in 1057 bytes} [Fri May 27 03:49:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:25.632962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.646271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a0786-ae31-4e1a-9da9-dfac2d6de0ab None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.709137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19d4aa-2fab-4cad-8e82-542194302aa5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.724673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.725623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.725918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.726204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.726933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.733977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:25.753297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.754950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.755420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.755802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.765833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:25.781762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.783751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:25.799060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 836/1668] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:25.839747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 833/1669] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:25 2022] POST /identity/v3/users => generated 284 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:25.843651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.854977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3cdf-604e-4028-a8a7-2afcbd4e0848 service placement] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.860740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.861650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.862076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.862493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.863361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.865190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:25.880333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 837/1670] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:25.884703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.898507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f486fb-2e21-4502-b3a9-f165e2486c03 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.904198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc51b-b5b2-49d0-8af3-556c68613571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.905335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc51b-b5b2-49d0-8af3-556c68613571 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.905915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc51b-b5b2-49d0-8af3-556c68613571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.906420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc51b-b5b2-49d0-8af3-556c68613571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.915031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcc51b-b5b2-49d0-8af3-556c68613571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.918110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcc51b-b5b2-49d0-8af3-556c68613571 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:25.932456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 834/1671] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:25.938093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:25.953263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95ce1f-af1a-42c6-b40f-00f79928ff84 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:25.960708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:25.960708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:25.960708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:25.961559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] PATH_INFO: `/v3/projects/684264526ead4bdcb1f49196c78f8a75/users/a1d772f001a14a1bb3cc96a22c870d1c/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:25.980681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=684264526ead4bdcb1f49196c78f8a75, user_id=a1d772f001a14a1bb3cc96a22c870d1c, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:25.984986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:26.066149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 838/1672] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:25 2022] PUT /identity/v3/projects/684264526ead4bdcb1f49196c78f8a75/users/a1d772f001a14a1bb3cc96a22c870d1c/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:26.080286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:26.081036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:26.081601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:26.148313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] MFA Rules not processed for user `a1d772f001a14a1bb3cc96a22c870d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:26.150115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:26.308275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 835/1673] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:26.313504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:26.471156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b115d-9440-4a2e-bfc3-3554b000e2dd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:26.487916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:26.488675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:26.488969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:26.489251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:26.490012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:26.491996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:26.624903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 839/1674] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:26 2022] POST /identity/v3/projects => generated 369 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:26.629023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:26.649155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10907b31-d390-41e6-ba84-c3f51a105d63 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:26.753204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:26.754316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:26.754611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:26.754902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:26.755734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:26.758724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:26.925946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 836/1675] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:26 2022] POST /identity/v3/users => generated 291 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:49:26.930042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:26.950144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691270f-a7bc-4406-8346-a6914d6037fb admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:26.974362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:26.974362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:26.974362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:26.974362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:26.974764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:26.975482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:26.977734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:26.996294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 840/1676] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:49:27.003752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:27.030151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f795e3d-0929-4de2-8f93-a4e5e560b92e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:27.049067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:27.054763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:27.054996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:27.055289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] PATH_INFO: `/v3/projects/7d35569facb54e90a8166d0a564cd9fa/users/7b0cae4c62b0496f8e25db811b798c5d/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:27.076345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:27.096644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d35569facb54e90a8166d0a564cd9fa, user_id=7b0cae4c62b0496f8e25db811b798c5d, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:27.107188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:27.215049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 837/1677] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:27 2022] PUT /identity/v3/projects/7d35569facb54e90a8166d0a564cd9fa/users/7b0cae4c62b0496f8e25db811b798c5d/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:27.219409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:27.230543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df07ee-05d8-4dda-85bf-bc34201cda0a admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:27.308406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:27.310301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:27.349691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:27.350388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:27.350680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:27.350963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:27.353937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:27.357340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:27.372545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 841/1678] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:27.377073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:27.389645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed60847-bed4-4c56-9021-d774752cff8b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:27.395845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a440b4-92b0-4534-a4a3-09b05df0ea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:27.396830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a440b4-92b0-4534-a4a3-09b05df0ea52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:27.397114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a440b4-92b0-4534-a4a3-09b05df0ea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:27.397539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a440b4-92b0-4534-a4a3-09b05df0ea52 admin admin] PATH_INFO: `/v3/projects/7d35569facb54e90a8166d0a564cd9fa/users/7b0cae4c62b0496f8e25db811b798c5d/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:27.401129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a440b4-92b0-4534-a4a3-09b05df0ea52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d35569facb54e90a8166d0a564cd9fa, user_id=7b0cae4c62b0496f8e25db811b798c5d, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:27.412121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a440b4-92b0-4534-a4a3-09b05df0ea52 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:27.423259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 842/1679] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:27 2022] PUT /identity/v3/projects/7d35569facb54e90a8166d0a564cd9fa/users/7b0cae4c62b0496f8e25db811b798c5d/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:27.434169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:27.434771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:27.435249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:27.470668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] MFA Rules not processed for user `7b0cae4c62b0496f8e25db811b798c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:27.481660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:27.485937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b370be5-ab6f-48b7-ab5d-aaf74e1795cb admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:27.600439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:27.602405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:27.603493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:27.605103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] PATH_INFO: `/v3/projects/024ccdd4520b48ef823b628e2a2ec915` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:27.606665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] RBAC: Authorizing `identity:delete_project(project_id=024ccdd4520b48ef823b628e2a2ec915)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:27.609436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:27.654948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 843/1680] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:27 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:27.671142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:27.672620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:27.820178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024ccdd4520b48ef823b628e2a2ec915', 'request_id': 'req-0bf90d52-cee3-44ea-8fe8-75e642a3329d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:27.838113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024ccdd4520b48ef823b628e2a2ec915', 'request_id': 'req-0bf90d52-cee3-44ea-8fe8-75e642a3329d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:27.847700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547609f3-9199-40ef-b15d-b885ed823254 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:27.946277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024ccdd4520b48ef823b628e2a2ec915', 'request_id': 'req-0bf90d52-cee3-44ea-8fe8-75e642a3329d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:27.993055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] The token cache is being invalidate because project 024ccdd4520b48ef823b628e2a2ec915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:27.995182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.002771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.012982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 838/1681] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:27 2022] DELETE /identity/v3/projects/024ccdd4520b48ef823b628e2a2ec915 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:28.014403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:28.020776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:28.058691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:28.062549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:28.069974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:28.070361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:28.079934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:28.227970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf90d52-cee3-44ea-8fe8-75e642a3329d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:28.238673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:28.240957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:28.241236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:28.241512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] PATH_INFO: `/v3/projects/13f8766f95b04e2e8ddd09600184e85a` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:28.245977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f8766f95b04e2e8ddd09600184e85a)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:28.248014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:28.258634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:28.258634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:28.281423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 844/1682] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:49:28.283456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:28.285912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:28.315756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527fdc-fa89-471b-a073-9357d25fac87 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:28.328694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:28.330307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:28.330802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:28.331334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] PATH_INFO: `/v3/users/508d7d006b7d46a78d0d9e534ddaefff` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:28.345544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508d7d006b7d46a78d0d9e534ddaefff)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:28.349411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:28.491114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f8766f95b04e2e8ddd09600184e85a', 'request_id': 'req-e149d7e1-e2be-43c0-b2d2-383f3efbe661'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.512791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f8766f95b04e2e8ddd09600184e85a', 'request_id': 'req-e149d7e1-e2be-43c0-b2d2-383f3efbe661'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.603095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f8766f95b04e2e8ddd09600184e85a', 'request_id': 'req-e149d7e1-e2be-43c0-b2d2-383f3efbe661'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.631977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] The token cache is being invalidate because project 13f8766f95b04e2e8ddd09600184e85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:28.641068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.643270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:28.646204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.648701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508d7d006b7d46a78d0d9e534ddaefff'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.649154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 839/1683] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:28 2022] DELETE /identity/v3/projects/13f8766f95b04e2e8ddd09600184e85a => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:28.650105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:28.651481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508d7d006b7d46a78d0d9e534ddaefff'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.652592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508d7d006b7d46a78d0d9e534ddaefff'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.652903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:28.686518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508d7d006b7d46a78d0d9e534ddaefff'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.705539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e92f8443-272c-4212-b9e5-65d6786e34a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508d7d006b7d46a78d0d9e534ddaefff'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:28.771852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149d7e1-e2be-43c0-b2d2-383f3efbe661 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:28.777474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:28.778413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:28.778701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:28.779030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:28.781650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:28.838322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 845/1684] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:28 2022] DELETE /identity/v3/users/508d7d006b7d46a78d0d9e534ddaefff => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:28.840294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:28.840751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:28.841157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:28.902660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:28.906391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:28.926674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:28.929702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f007ab2-4644-4420-8604-5210d2b887c8 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:28.956041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 840/1685] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:29.074908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 846/1686] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:29.079660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.080132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.080490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.103117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:29.133086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:29.142179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:29.176034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 841/1687] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:29.180738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:29.195693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec080c-1625-4d0f-8e99-c6590cb2fdfa None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:29.212690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:29.213367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.213655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.213933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.214673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:29.214978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:29.217374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:29.235784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 847/1688] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:29.242600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:29.257903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fe791-adcc-4b0b-8dc5-b5e9edddf7d3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:29.262333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:29.263329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.264342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.264873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.265780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:29.267910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:29.399897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 842/1689] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:29 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:29.403966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:29.418709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543097e1-c0df-42f1-a7c8-951146670e9c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:29.570513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:29.571888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.572182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.573666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.574458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:29.580769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:29.673205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 848/1690] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:29 2022] POST /identity/v3/users => generated 296 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:29.677384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:29.691362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c2e9e-cb65-483d-9356-212db00a1b5e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:29.710054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:29.710993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.711289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.714017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.716517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:29.723832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:29.743533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 843/1691] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:29.747712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:29.774456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866064d-7e5a-4bf3-91da-5ed5839c3115 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:29.784354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44eebba-96e0-400a-b12c-22475fdcef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:29.791620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eebba-96e0-400a-b12c-22475fdcef2d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.791620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eebba-96e0-400a-b12c-22475fdcef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.791620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44eebba-96e0-400a-b12c-22475fdcef2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.791620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eebba-96e0-400a-b12c-22475fdcef2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:29.791620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44eebba-96e0-400a-b12c-22475fdcef2d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:29.837895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 849/1692] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:29.842983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:29.865407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6648762-a04d-4c14-885e-e12d0b18c9fb admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:29.887768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:29.889753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:29.890259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:29.890804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] PATH_INFO: `/v3/projects/b3ce8c1e18c247ec8d943c75e5ed4af9/users/75220159999b4f7780ccd2472e594c1b/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:29.976935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ce8c1e18c247ec8d943c75e5ed4af9, user_id=75220159999b4f7780ccd2472e594c1b, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:30.000214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:30.165523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 844/1693] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:29 2022] PUT /identity/v3/projects/b3ce8c1e18c247ec8d943c75e5ed4af9/users/75220159999b4f7780ccd2472e594c1b/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:30.176542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:30.177089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:30.177268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:30.266659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] MFA Rules not processed for user `75220159999b4f7780ccd2472e594c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:30.268686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:30.358858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 850/1694] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:30.363164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:30.437124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:30.439922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:30.460223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ad38b-69a4-42ed-80ec-c1851ede6c82 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:30.467515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:30.469653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:30.470162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:30.470899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:30.472568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:30.476542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:30.653775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 845/1695] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:30 2022] POST /identity/v3/projects => generated 389 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:30.655469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:30.657966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690bb41-85c6-4c90-b927-208a228594d1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:30.660967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:30.831685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:30.852259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:30.853178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:30.853459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:30.853734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] PATH_INFO: `/v3/projects/e2b2d69babfc451ea96be5c9938ce90e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:30.861058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b2d69babfc451ea96be5c9938ce90e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:30.871919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:30.913989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e8e8b-8e17-4f10-8eda-a8ad523e5531 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:30.945703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:30.946413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:30.954617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:30.961498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:30.965336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:30.971619 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:31.133469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b2d69babfc451ea96be5c9938ce90e', 'request_id': 'req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:31.140782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 846/1696] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:30 2022] POST /identity/v3/users => generated 301 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:31.146981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:31.175442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb560a33-b6de-48c8-b56a-45cdab2f2b64 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:31.226350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b2d69babfc451ea96be5c9938ce90e', 'request_id': 'req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:31.234244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b2d69babfc451ea96be5c9938ce90e', 'request_id': 'req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:31.259075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] The token cache is being invalidate because project e2b2d69babfc451ea96be5c9938ce90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:31.259374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:31.262667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b1e7c9e-1f70-41bd-bb92-84e62cb80a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:31.269184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 851/1697] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:30 2022] DELETE /identity/v3/projects/e2b2d69babfc451ea96be5c9938ce90e => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:31.283647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cae8f-c6eb-41fb-9b45-90d9e180137b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:31.283944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cae8f-c6eb-41fb-9b45-90d9e180137b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:31.284232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cae8f-c6eb-41fb-9b45-90d9e180137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:31.333519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0e5cae8f-c6eb-41fb-9b45-90d9e180137b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:31.334756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0e5cae8f-c6eb-41fb-9b45-90d9e180137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:31.375919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:31.377143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:31.377588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:31.380138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:31.381257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:31.392706 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:31.433701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 847/1698] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:31.439094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:31.446385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:31.482026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6231210-bb1e-4f03-a1c9-4fd7ac961e82 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:31.510645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 852/1699] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:31.512219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:31.512909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:31.513264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497f24c-2eb3-447e-8ac3-cb3eaf0f8fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:31.513788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:31.514987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f24c-2eb3-447e-8ac3-cb3eaf0f8fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:31.515263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f24c-2eb3-447e-8ac3-cb3eaf0f8fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:31.515538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f24c-2eb3-447e-8ac3-cb3eaf0f8fa1 admin admin] PATH_INFO: `/v3/projects/f312fdd45e2f482c88b92a898ea956d3/users/7cedf32edfe24fd9b79c2fd783ef4d8e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:31.557100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:31.558418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:31.584444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497f24c-2eb3-447e-8ac3-cb3eaf0f8fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f312fdd45e2f482c88b92a898ea956d3, user_id=7cedf32edfe24fd9b79c2fd783ef4d8e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:31.587460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497f24c-2eb3-447e-8ac3-cb3eaf0f8fa1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:31.677675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 848/1700] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:31 2022] PUT /identity/v3/projects/f312fdd45e2f482c88b92a898ea956d3/users/7cedf32edfe24fd9b79c2fd783ef4d8e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:31.679147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:31.679580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:31.679993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:31.718194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 853/1701] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:49:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:31.719329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:31.754769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7fbc6-54c1-440f-97c9-4beabbed33b1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:31.769703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:31.771298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:31.788099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:31.939294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 849/1702] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:31.954617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:31.962214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:31.962978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:31.963266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:31.963552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:31.964293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:31.966201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:31.993231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 854/1703] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:31.996852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:31.999893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c94515-e4b0-4050-9f75-e883ec5be9ea None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.010003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.011035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.011463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.011882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.012791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:32.017665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:32.019585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:32.023526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df966bb-3acc-4665-bc29-41feda3717d2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.032284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5dae-d6ca-4182-b897-41a43a6a6d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.038344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5dae-d6ca-4182-b897-41a43a6a6d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.038344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5dae-d6ca-4182-b897-41a43a6a6d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.038344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5dae-d6ca-4182-b897-41a43a6a6d2d admin admin] PATH_INFO: `/v3/projects/f312fdd45e2f482c88b92a898ea956d3/users/7cedf32edfe24fd9b79c2fd783ef4d8e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.043869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da5dae-d6ca-4182-b897-41a43a6a6d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f312fdd45e2f482c88b92a898ea956d3, user_id=7cedf32edfe24fd9b79c2fd783ef4d8e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:32.053813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da5dae-d6ca-4182-b897-41a43a6a6d2d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:32.059893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 850/1704] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:32.061914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:32.103474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 855/1705] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:31 2022] PUT /identity/v3/projects/f312fdd45e2f482c88b92a898ea956d3/users/7cedf32edfe24fd9b79c2fd783ef4d8e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:32.107189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87ab2d-9482-431c-b918-1a2d5d7c4af0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.114566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.114856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.115139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.198079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] MFA Rules not processed for user `7cedf32edfe24fd9b79c2fd783ef4d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:32.199367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:32.278327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.279023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.282481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.282832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.283574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:32.285555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:32.420489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 851/1706] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:32 2022] POST /identity/v3/projects => generated 369 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:32.421734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:32.426605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:32.432491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 856/1707] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:32.436404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:32.464653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a28b1-7b70-4fde-8a8a-b0ff96ff0e7d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.584265 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f08c-32c0-4367-9904-7a90de0869a9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.592449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.594397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.594674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.595573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.613388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.614245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.614550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.614835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.617155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:32.620662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:32.705425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 857/1708] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:32 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:32.708145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:32.710954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:32.713452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:32.738083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:32.739779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:32.772884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 852/1709] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:49:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:49:32.773647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:32.809557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9a92-a904-4c3a-bdba-e4c2cbd17aa2 service swift] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.845417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.846379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.846837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.847291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.848169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:32.850170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd65e40-edc7-49b5-aec3-f179c043664d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:32.859759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:32.862928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38820c-de07-4d68-8bd4-d0b89bb2ad9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:32.863687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38820c-de07-4d68-8bd4-d0b89bb2ad9e service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:32.864128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38820c-de07-4d68-8bd4-d0b89bb2ad9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:32.864525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38820c-de07-4d68-8bd4-d0b89bb2ad9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:32.930456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 853/1710] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:32.934556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:32.989340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143c4bb-66e3-411e-be5e-cc4326427f4d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:33.018991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:33.020018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:33.020525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:33.021225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:33.022214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:33.024604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:33.041769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 854/1711] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:33.048753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:33.063894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d8462-6d58-4fa9-97c7-edb6d63f4378 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:33.106785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:33.108336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38820c-de07-4d68-8bd4-d0b89bb2ad9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:33.108494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:33.116678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:33.116678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] PATH_INFO: `/v3/projects/3d42d2bc219f43928a64b620669fe948/users/9e0ce3803d8f457480ccf0bafb1d0671/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:33.121651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38820c-de07-4d68-8bd4-d0b89bb2ad9e service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:33.178633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d42d2bc219f43928a64b620669fe948, user_id=9e0ce3803d8f457480ccf0bafb1d0671, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:33.184593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:33.221003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 858/1712] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:33.311554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 855/1713] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:33 2022] PUT /identity/v3/projects/3d42d2bc219f43928a64b620669fe948/users/9e0ce3803d8f457480ccf0bafb1d0671/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:33.312064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:33.312381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:33.312726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:33.385280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:33.387823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] MFA Rules not processed for user `9e0ce3803d8f457480ccf0bafb1d0671`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:33.388240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:33.390047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:33.482994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 859/1714] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:33.489247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:33.653234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16546352-5157-4593-81a1-bfb04c2f5d1f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:33.680005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:33.682995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:33.683344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:33.683630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:33.700376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:33.785241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:33.831413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b8fc5-28d1-4433-a487-30bffe24b9d6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:33.864202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:33.866422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:33.887047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:33.887481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:33.887778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:33.889164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:33.891412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:34.011079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:34.012530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:34.067005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 856/1715] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:49:34.182751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 860/1716] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:33 2022] POST /identity/v3/projects => generated 367 bytes in 691 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:34.184091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:34.208100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc2428-9149-4096-bc59-4681938ad2c9 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:34.301466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:34.618878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:34.618878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:34.618878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:34.618878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:34.621207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:34.621207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:34.709005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:34.974165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 857/1717] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:34 2022] POST /identity/v3/users => generated 290 bytes in 790 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:34.977909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:35.025011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c3ace-5416-4cac-8ee8-ff0b2c0f927c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:35.073176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:35.082055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:35.085119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:35.086203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:35.095806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:35.124307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:35.174032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 861/1718] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:35.179341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:35.199083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5a9ae-2d36-42f5-b087-38b2235167d1 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:35.214213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:35.218959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:35.219505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:35.220923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] PATH_INFO: `/v3/projects/f66bd6ae98a14ff7bbe3c703a0c4cbb6/users/1fd2f791559f47b8b99e9bec74fe3b1b/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:35.293007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66bd6ae98a14ff7bbe3c703a0c4cbb6, user_id=1fd2f791559f47b8b99e9bec74fe3b1b, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:35.316328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:35.424662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 858/1719] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:35 2022] PUT /identity/v3/projects/f66bd6ae98a14ff7bbe3c703a0c4cbb6/users/1fd2f791559f47b8b99e9bec74fe3b1b/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:35.426998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:35.448874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62153a20-c6b0-4355-ab11-8ec21b1ff548 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:35.738036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd76239-8601-4eb9-8984-ca9c7d4b9f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:35.738733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd76239-8601-4eb9-8984-ca9c7d4b9f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:35.739019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd76239-8601-4eb9-8984-ca9c7d4b9f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:35.739301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd76239-8601-4eb9-8984-ca9c7d4b9f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:35.740025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd76239-8601-4eb9-8984-ca9c7d4b9f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:35.762258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd76239-8601-4eb9-8984-ca9c7d4b9f40 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:35.833067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 862/1720] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:35.833247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:35.864066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe9132-59a3-43a3-b2c2-369892b42927 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:35.881968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:35.882532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:35.882992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:35.890243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:35.934577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207a76d-c5b4-4dca-bdd5-faa15798a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:35.939664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207a76d-c5b4-4dca-bdd5-faa15798a3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:35.944140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207a76d-c5b4-4dca-bdd5-faa15798a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:35.944487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207a76d-c5b4-4dca-bdd5-faa15798a3dd admin admin] PATH_INFO: `/v3/projects/f66bd6ae98a14ff7bbe3c703a0c4cbb6/users/1fd2f791559f47b8b99e9bec74fe3b1b/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:35.959448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207a76d-c5b4-4dca-bdd5-faa15798a3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66bd6ae98a14ff7bbe3c703a0c4cbb6, user_id=1fd2f791559f47b8b99e9bec74fe3b1b, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:35.963698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207a76d-c5b4-4dca-bdd5-faa15798a3dd admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:35.977828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:36.037274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:36.063073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 859/1721] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:35 2022] PUT /identity/v3/projects/f66bd6ae98a14ff7bbe3c703a0c4cbb6/users/1fd2f791559f47b8b99e9bec74fe3b1b/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:36.084531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:36.086927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:36.088244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:36.215359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] MFA Rules not processed for user `1fd2f791559f47b8b99e9bec74fe3b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:36.241522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:36.306495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 863/1722] 10.222.0.44 () {58 vars in 1015 bytes} [Fri May 27 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:36.378153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:36.388201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:36.429491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1670d8c-e8ea-414d-9f90-e588f68ae22d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:36.434342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb143-abbd-4de5-b7f6-062b9fd1e04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:36.435208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb143-abbd-4de5-b7f6-062b9fd1e04c service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:36.444586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb143-abbd-4de5-b7f6-062b9fd1e04c service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:36.446311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb143-abbd-4de5-b7f6-062b9fd1e04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:36.482027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddb143-abbd-4de5-b7f6-062b9fd1e04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:36.517173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:36.523910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddb143-abbd-4de5-b7f6-062b9fd1e04c service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:36.558128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 864/1723] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:36.562765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:36.563451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:36.564008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:36.588534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 860/1724] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:36.616829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:36.651335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] MFA Rules not processed for user `bca16fe930244e12bbd12a23323d583e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:36.652940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:36.817333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebf33b-0324-4d3e-9303-f98ff713afad None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:36.827241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 865/1725] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:36.831587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1155769-022d-4d47-be4b-7b89aeece9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:36.832602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1155769-022d-4d47-be4b-7b89aeece9bd service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:36.832891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1155769-022d-4d47-be4b-7b89aeece9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:36.833166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1155769-022d-4d47-be4b-7b89aeece9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:36.842829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:36.844750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:36.859168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:36.881420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1155769-022d-4d47-be4b-7b89aeece9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:36.882886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1155769-022d-4d47-be4b-7b89aeece9bd service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:36.922275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 861/1726] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:37.075785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1b548-a170-45b9-b890-48080cde92b8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:37.098951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:37.099242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:37.099522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:37.105185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe5b97-b7c2-435d-bafd-8b63f2b46191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:37.105873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe5b97-b7c2-435d-bafd-8b63f2b46191 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:37.106161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe5b97-b7c2-435d-bafd-8b63f2b46191 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:37.112439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe5b97-b7c2-435d-bafd-8b63f2b46191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:37.165066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe5b97-b7c2-435d-bafd-8b63f2b46191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:37.166694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe5b97-b7c2-435d-bafd-8b63f2b46191 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:37.186241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] MFA Rules not processed for user `65c434b614e34b1f9b79375d85720e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:37.187364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:37.211742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 866/1727] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:37.514158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 862/1728] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:37.524590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:37.524894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:37.525250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:37.601950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:37.606622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:37.627750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] MFA Rules not processed for user `65c434b614e34b1f9b79375d85720e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:37.629045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:37.692100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 867/1729] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:37.703895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:37.707233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:37.756408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:37.934695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df6aa5-baa7-45ce-8ea1-3acbab8e194c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:37.970765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:37.973252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:37.976014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:37.976299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] PATH_INFO: `/v3/users/a1d772f001a14a1bb3cc96a22c870d1c` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:37.991739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d772f001a14a1bb3cc96a22c870d1c)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:37.991739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:38.003400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9cd48-1b96-4963-bfd7-26328183e868 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:38.021949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:38.023157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:38.027397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:38.027937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] PATH_INFO: `/v3/users/df05bff695ce4889bc597c9437142799` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:38.031866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=df05bff695ce4889bc597c9437142799)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:38.037299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:38.151400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:38.255832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d772f001a14a1bb3cc96a22c870d1c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.256962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d772f001a14a1bb3cc96a22c870d1c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.274999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d772f001a14a1bb3cc96a22c870d1c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.275801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d772f001a14a1bb3cc96a22c870d1c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.321753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df05bff695ce4889bc597c9437142799'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.321753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df05bff695ce4889bc597c9437142799'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.321753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df05bff695ce4889bc597c9437142799'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.358295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df05bff695ce4889bc597c9437142799'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.377356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435feda6-2044-4200-8ee3-1c5ec6c923cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df05bff695ce4889bc597c9437142799'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.481536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d772f001a14a1bb3cc96a22c870d1c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:38.522955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 863/1730] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:37 2022] DELETE /identity/v3/users/a1d772f001a14a1bb3cc96a22c870d1c => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:38.523843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:38.535301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:38.603796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 868/1731] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:37 2022] DELETE /identity/v3/users/df05bff695ce4889bc597c9437142799 => generated 0 bytes in 898 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:38.605497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85717-07b1-4f9a-b19e-9a80706b3bdc None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:38.605791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85717-07b1-4f9a-b19e-9a80706b3bdc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:38.606073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c85717-07b1-4f9a-b19e-9a80706b3bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:38.681651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-29c85717-07b1-4f9a-b19e-9a80706b3bdc None None] MFA Rules not processed for user `3cfb66d6bc254e549d6d355c2bbb440a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:38.687866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-29c85717-07b1-4f9a-b19e-9a80706b3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:38.840786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:38.849942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 869/1732] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:38.851604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251912-bede-4ab7-82ab-4e6ffbc7b1f3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:38.864067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251912-bede-4ab7-82ab-4e6ffbc7b1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:38.869949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59f77-a607-4252-b9a9-62d975b7bbe3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:38.870143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac251912-bede-4ab7-82ab-4e6ffbc7b1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:38.938674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ac251912-bede-4ab7-82ab-4e6ffbc7b1f3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:38.939917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ac251912-bede-4ab7-82ab-4e6ffbc7b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:39.090064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:39.105977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:39.105977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:39.105977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:39.105977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:39.257892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 870/1733] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:39.261801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:39.262355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:39.262885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:39.282208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:39.299978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:39.302607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:39.315881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 871/1734] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:39.317290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:39.326710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:39.331006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:39.338034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:39.359481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:39.363016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 864/1735] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 839 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:39.364458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:39.366111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:39.411704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66b87e-e56b-4548-b859-404310d12401 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:39.435010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:39.437158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:39.438663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:39.439199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:39.465686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:39.468213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:39.518754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 872/1736] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:39.535639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e301a75-df3d-4ba1-94c0-79948d08539d service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:39.540541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:39.555129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f71c2-b7e7-41e0-85e2-506ec74bc8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:39.555798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f71c2-b7e7-41e0-85e2-506ec74bc8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:39.556077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f71c2-b7e7-41e0-85e2-506ec74bc8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:39.556352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f71c2-b7e7-41e0-85e2-506ec74bc8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:39.588040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f71c2-b7e7-41e0-85e2-506ec74bc8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:39.589530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f71c2-b7e7-41e0-85e2-506ec74bc8f8 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:39.610591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 865/1737] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:39.625544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1d31c-48b5-4fb5-91cf-e1812b663859 service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:39.646478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1420057-3fa4-472a-b2d3-a4260daf3846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:39.648871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420057-3fa4-472a-b2d3-a4260daf3846 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:39.654565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420057-3fa4-472a-b2d3-a4260daf3846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:39.654948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1420057-3fa4-472a-b2d3-a4260daf3846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:39.695036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1420057-3fa4-472a-b2d3-a4260daf3846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:39.703666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1420057-3fa4-472a-b2d3-a4260daf3846 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:39.780794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 873/1738] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:40.105249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb8866-7553-446f-a61a-4052b76ac6ab None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:40.106497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb8866-7553-446f-a61a-4052b76ac6ab None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:40.106922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb8866-7553-446f-a61a-4052b76ac6ab None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:40.109681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 866/1739] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:49:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:40.310585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33267dd7-8303-4da5-8d8a-e05ea886ea96 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:40.311262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33267dd7-8303-4da5-8d8a-e05ea886ea96 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:40.311668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33267dd7-8303-4da5-8d8a-e05ea886ea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:40.349539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-33267dd7-8303-4da5-8d8a-e05ea886ea96 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:40.351677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-33267dd7-8303-4da5-8d8a-e05ea886ea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:40.390714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 874/1740] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:49:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:41.235491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb8866-7553-446f-a61a-4052b76ac6ab None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:41.317347 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb8866-7553-446f-a61a-4052b76ac6ab None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:41.348768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dbb4b-9fe7-4828-9a28-b5a9bb69b1cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:41.352376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dbb4b-9fe7-4828-9a28-b5a9bb69b1cf service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:41.353064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dbb4b-9fe7-4828-9a28-b5a9bb69b1cf service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:41.353548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dbb4b-9fe7-4828-9a28-b5a9bb69b1cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:41.391924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dbb4b-9fe7-4828-9a28-b5a9bb69b1cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:41.393896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dbb4b-9fe7-4828-9a28-b5a9bb69b1cf service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:41.451355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 867/1741] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:41.682040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b5c4-63d5-42aa-9d7e-ecbc3a815a17 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:41.683194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b5c4-63d5-42aa-9d7e-ecbc3a815a17 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:41.683687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b5c4-63d5-42aa-9d7e-ecbc3a815a17 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:41.685675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 875/1742] 10.222.0.58 () {54 vars in 912 bytes} [Fri May 27 03:49:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:49:41.699899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:41.699899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:41.700225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:41.756068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:41.758602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:41.785887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 868/1743] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:41.957084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b5c4-63d5-42aa-9d7e-ecbc3a815a17 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:41.981454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b5c4-63d5-42aa-9d7e-ecbc3a815a17 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:41.995062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:41.997063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:42.001787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:42.002276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:42.047213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:42.049899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:42.067497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 876/1744] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:42.428539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:42.430107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:42.485262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c2aee-17cd-41e4-a010-5a343c7d65a3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:42.512726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:42.515126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:42.516549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:42.517871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] PATH_INFO: `/v3/projects/684264526ead4bdcb1f49196c78f8a75` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:42.521009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=684264526ead4bdcb1f49196c78f8a75)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:42.526977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:42.578028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:42.579738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:42.620663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da95c2-0082-4d22-945b-52419ec64357 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:42.637247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:42.643023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:42.643573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:42.644051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] PATH_INFO: `/v3/users/75220159999b4f7780ccd2472e594c1b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:42.650297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75220159999b4f7780ccd2472e594c1b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:42.652619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:42.790761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684264526ead4bdcb1f49196c78f8a75', 'request_id': 'req-5092bd68-5f72-4d86-9362-0ceb100b26a3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.814874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684264526ead4bdcb1f49196c78f8a75', 'request_id': 'req-5092bd68-5f72-4d86-9362-0ceb100b26a3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.962444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75220159999b4f7780ccd2472e594c1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.967316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75220159999b4f7780ccd2472e594c1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.967316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75220159999b4f7780ccd2472e594c1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.972267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684264526ead4bdcb1f49196c78f8a75', 'request_id': 'req-5092bd68-5f72-4d86-9362-0ceb100b26a3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.992946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] The token cache is being invalidate because project 684264526ead4bdcb1f49196c78f8a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:42.993851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.995884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.998833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75220159999b4f7780ccd2472e594c1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:42.999615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 869/1745] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:42 2022] DELETE /identity/v3/projects/684264526ead4bdcb1f49196c78f8a75 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:43.004586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:43.004586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:43.028650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75220159999b4f7780ccd2472e594c1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.085344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:43.132624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092bd68-5f72-4d86-9362-0ceb100b26a3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:43.140328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 877/1746] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:42 2022] DELETE /identity/v3/users/75220159999b4f7780ccd2472e594c1b => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:43.141594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:43.144481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:43.145487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:43.145787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:43.146067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] PATH_INFO: `/v3/users/9e0ce3803d8f457480ccf0bafb1d0671` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:43.146485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:43.147465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0ce3803d8f457480ccf0bafb1d0671)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:43.151912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:43.305926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538240cb-c6b0-43c7-b7d0-f4675d744c12 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:43.317252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:43.318274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:43.318581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:43.318875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] PATH_INFO: `/v3/users/7b0cae4c62b0496f8e25db811b798c5d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:43.324020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0cae4c62b0496f8e25db811b798c5d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:43.326289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:43.371830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:43.507543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0ce3803d8f457480ccf0bafb1d0671'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.510611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0ce3803d8f457480ccf0bafb1d0671'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.528318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0ce3803d8f457480ccf0bafb1d0671'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.529643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0ce3803d8f457480ccf0bafb1d0671'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.608146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0cae4c62b0496f8e25db811b798c5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.609010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0cae4c62b0496f8e25db811b798c5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.609781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0cae4c62b0496f8e25db811b798c5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.618629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e9eafa37-2238-4bb7-b2e5-d0cbc3af3e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0ce3803d8f457480ccf0bafb1d0671'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.658192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0cae4c62b0496f8e25db811b798c5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.663124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 870/1747] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:42 2022] DELETE /identity/v3/users/9e0ce3803d8f457480ccf0bafb1d0671 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:43.667241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:43.667542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:43.667919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:43.688405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81df1c17-ec0b-4205-a7b5-71ff9928f1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0cae4c62b0496f8e25db811b798c5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:43.713818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:43.717046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:43.841548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 878/1748] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:43 2022] DELETE /identity/v3/users/7b0cae4c62b0496f8e25db811b798c5d => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:43.858475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:43.879347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:43.884256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:43.939331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 871/1749] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:43.948184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:43.952167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:43.973741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:43.976998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:44.134851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 879/1750] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:44.176308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3ec41-9281-4bb0-96a8-ac68a4fff365 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:44.196752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:44.224843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:44.328840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c96d-5c4d-41f2-a97c-9a0d9bb67076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:44.330440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c96d-5c4d-41f2-a97c-9a0d9bb67076 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:44.331019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c96d-5c4d-41f2-a97c-9a0d9bb67076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:44.331493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c96d-5c4d-41f2-a97c-9a0d9bb67076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:44.347675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2d509-0716-4d69-aa3d-43b5f9d62128 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:44.355927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:44.356647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:44.356932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:44.358069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:44.359925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c96d-5c4d-41f2-a97c-9a0d9bb67076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:44.361632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c96d-5c4d-41f2-a97c-9a0d9bb67076 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:44.375504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 872/1751] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:44.385201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:44.386812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:44.413032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 880/1752] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:49:44.467730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb762e-bfaa-42bc-b0a4-baeae06d23f2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:44.468138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb762e-bfaa-42bc-b0a4-baeae06d23f2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:44.468481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb762e-bfaa-42bc-b0a4-baeae06d23f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:44.508857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-14eb762e-bfaa-42bc-b0a4-baeae06d23f2 None None] MFA Rules not processed for user `65c434b614e34b1f9b79375d85720e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:44.515989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-14eb762e-bfaa-42bc-b0a4-baeae06d23f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:44.714355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 873/1753] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:44.784278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:44.786063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:44.891268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b3d5f-c7af-4e56-8325-1eca67018653 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:44.923283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:44.924206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:44.924488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:44.924957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] PATH_INFO: `/v3/projects/4879215a268c47c782a6400a394b3968` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:44.928426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4879215a268c47c782a6400a394b3968)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:44.930422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:45.069218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:45.099636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4879215a268c47c782a6400a394b3968', 'request_id': 'req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.202150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4879215a268c47c782a6400a394b3968', 'request_id': 'req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.209799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4879215a268c47c782a6400a394b3968', 'request_id': 'req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.236675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] The token cache is being invalidate because project 4879215a268c47c782a6400a394b3968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:45.237065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.237700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0ccb3d0f-a369-4807-bae0-243c8d68a7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.240553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 881/1754] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:44 2022] DELETE /identity/v3/projects/4879215a268c47c782a6400a394b3968 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:45.245426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:45.245763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:45.246039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:45.287870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:45.289479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:45.503147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 874/1755] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:45.504695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:45.505013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:45.505393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:45.522893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:45.525180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:45.542170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc9424-a608-4384-992e-0b97886cae97 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:45.551654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:45.553055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:45.577162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 882/1756] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:45.581136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:45.598090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa6075-4009-444a-81cf-e6cab50c3b82 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:45.610760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:45.612271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:45.613472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:45.613665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:45.613917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:45.614201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:45.614959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:45.615464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:45.617448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] PATH_INFO: `/v3/projects/7d35569facb54e90a8166d0a564cd9fa` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:45.617573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:45.619178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:45.626424 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d35569facb54e90a8166d0a564cd9fa)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:45.629477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:45.658413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 883/1757] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:49:45.666706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:45.688059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e9648-1825-405b-b8c6-9a0193dfbffe admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:45.701481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:45.702257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:45.702588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:45.702870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:45.703624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:45.705734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:45.859179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 884/1758] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:45 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:49:45.871038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:45.901663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d35569facb54e90a8166d0a564cd9fa', 'request_id': 'req-6c27037e-770f-4934-a2d8-91674bcdb782'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.933876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d35569facb54e90a8166d0a564cd9fa', 'request_id': 'req-6c27037e-770f-4934-a2d8-91674bcdb782'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:45.956969 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:46.047213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d35569facb54e90a8166d0a564cd9fa', 'request_id': 'req-6c27037e-770f-4934-a2d8-91674bcdb782'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:46.067132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7be23-3067-4947-9c3e-a048fbfa2863 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.069979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] The token cache is being invalidate because project 7d35569facb54e90a8166d0a564cd9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:46.070280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:46.073637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6c27037e-770f-4934-a2d8-91674bcdb782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:46.079227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 875/1759] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:45 2022] DELETE /identity/v3/projects/7d35569facb54e90a8166d0a564cd9fa => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:46.082788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d0d98-4fde-4f7f-bf94-2aabd9698db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.083762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0d98-4fde-4f7f-bf94-2aabd9698db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.084111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.084453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.084666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0d98-4fde-4f7f-bf94-2aabd9698db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.084764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.085155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d0d98-4fde-4f7f-bf94-2aabd9698db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.086260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d0d98-4fde-4f7f-bf94-2aabd9698db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.088519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d0d98-4fde-4f7f-bf94-2aabd9698db1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:46.176007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:46.180724 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:46.227576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 876/1760] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:46.235180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 885/1761] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:45 2022] POST /identity/v3/users => generated 291 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:46.235970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.237764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:46.241474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.241474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.241710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.282664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761afd66-9900-4821-9dd7-b21e759de911 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.294316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:46.297315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:46.299363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.306038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.306398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.306683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.308595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.312304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:46.331182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 886/1762] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:46.333584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.347689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b49a7-a268-48b6-a8ff-5ef7976a61ec None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.366730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 877/1763] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:49:46.369046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.379558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.380384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.380798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.381223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.382203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:46.382680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.387425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:46.405456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4543-c23e-4d10-bc1e-8aa6d4bd6f17 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.415033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 887/1764] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:46.419909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.422536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.430882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.430882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.431200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.433609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.436928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4a6b-e0b0-4557-af4e-2b0c1a7e2f49 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.442092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:46.449232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.451844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.452561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.454670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.458224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.466023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:46.493350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 878/1765] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:46.495396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.517998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c2571-8faa-4843-b6ac-66d679bd76de admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.539853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ac4a6-962e-49a8-8deb-f5bae01038be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.543782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ac4a6-962e-49a8-8deb-f5bae01038be admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.546563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ac4a6-962e-49a8-8deb-f5bae01038be admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.549091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ac4a6-962e-49a8-8deb-f5bae01038be admin admin] PATH_INFO: `/v3/projects/759443049094408189e8bf4daebea4ea/users/947d7ba6b4cd443fbb6e8733ed2706f1/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.601765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ac4a6-962e-49a8-8deb-f5bae01038be admin admin] RBAC: Authorizing `identity:create_grant(project_id=759443049094408189e8bf4daebea4ea, user_id=947d7ba6b4cd443fbb6e8733ed2706f1, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.609728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ac4a6-962e-49a8-8deb-f5bae01038be admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:46.620590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 888/1766] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:46 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:46.622998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.686661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:46.702571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 879/1767] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:46 2022] PUT /identity/v3/projects/759443049094408189e8bf4daebea4ea/users/947d7ba6b4cd443fbb6e8733ed2706f1/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:46.718797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.718797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.718797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.782816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] MFA Rules not processed for user `947d7ba6b4cd443fbb6e8733ed2706f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:46.784788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:46.805203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58d39-3d1d-446a-b354-0ff81d596cad admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:46.975662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:46.977165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:46.977282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:46.977553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:46.978403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:46.979294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 880/1768] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:46.983753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:46.984654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:47.004422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5c7a-971b-4204-a04b-d12f18f45f0d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:47.011153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:47.012319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:47.012826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:47.013219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:47.014887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:47.018620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:47.176901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 889/1769] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:46 2022] POST /identity/v3/users => generated 284 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:47.177565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:47.211500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 881/1770] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:46 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:47.214549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:47.258704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba08017-c701-47d7-b822-df6498246660 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:47.371430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13158a-3365-4faa-bccf-55cf400a4bae admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:47.400710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:47.404731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:47.404731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:47.404731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:47.404731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:47.408754 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:47.436074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:47.436804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:47.438279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:47.438579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:47.440669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:47.442823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:47.452895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:47.530321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 890/1771] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:47.532072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:47.541196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:47.558097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 882/1772] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:47 2022] POST /identity/v3/users => generated 296 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:47.559140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:47.568957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:47.590644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b43ab-fde8-484e-92e9-b28521feff8b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:47.600093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:47.604721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:47.604721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:47.604721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] PATH_INFO: `/v3/projects/3d42d2bc219f43928a64b620669fe948` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:47.604721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d42d2bc219f43928a64b620669fe948)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:47.624245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:47.658225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044ab38-0e2f-4061-a877-bdc40b2d8440 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:47.682542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:47.687781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:47.688121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:47.688412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] PATH_INFO: `/v3/projects/b3ce8c1e18c247ec8d943c75e5ed4af9` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:47.693100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ce8c1e18c247ec8d943c75e5ed4af9)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:47.702668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:47.882262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d42d2bc219f43928a64b620669fe948', 'request_id': 'req-a543d1ae-5581-4ab5-911c-aa454388ef6e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.890677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:47.938413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d42d2bc219f43928a64b620669fe948', 'request_id': 'req-a543d1ae-5581-4ab5-911c-aa454388ef6e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.941640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ce8c1e18c247ec8d943c75e5ed4af9', 'request_id': 'req-8888422f-ff26-4cad-8f6e-6c282d69f41c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.956435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ce8c1e18c247ec8d943c75e5ed4af9', 'request_id': 'req-8888422f-ff26-4cad-8f6e-6c282d69f41c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.958178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d42d2bc219f43928a64b620669fe948', 'request_id': 'req-a543d1ae-5581-4ab5-911c-aa454388ef6e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.980305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] The token cache is being invalidate because project 3d42d2bc219f43928a64b620669fe948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:47.981194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.982692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a543d1ae-5581-4ab5-911c-aa454388ef6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:47.985046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 891/1773] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:47 2022] DELETE /identity/v3/projects/3d42d2bc219f43928a64b620669fe948 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:47.986979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:47.987463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:47.987741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:48.013896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] MFA Rules not processed for user `bca16fe930244e12bbd12a23323d583e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:48.015653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:48.040831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ce8c1e18c247ec8d943c75e5ed4af9', 'request_id': 'req-8888422f-ff26-4cad-8f6e-6c282d69f41c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.067653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] The token cache is being invalidate because project b3ce8c1e18c247ec8d943c75e5ed4af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:48.067953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.072121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.074572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 883/1774] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:47 2022] DELETE /identity/v3/projects/b3ce8c1e18c247ec8d943c75e5ed4af9 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:48.075788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:48.084153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:48.167314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 892/1775] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:48.168004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:48.169503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:48.311648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0811a9c-f898-451f-b8b3-102b7fa3b192 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:48.338508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888422f-ff26-4cad-8f6e-6c282d69f41c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:48.345927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:48.346650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:48.352346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:48.352868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:48.353913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:48.361672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:48.364746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:48.370327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:48.370951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:48.371341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:48.372374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:48.399175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:48.407555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 893/1776] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:48.409147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:48.415172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:48.434937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 884/1777] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:48.437443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:48.446114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:48.452054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2687a-290b-48a1-b275-b35a5609b9b1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:48.472975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:48.481005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:48.481005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:48.481005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] PATH_INFO: `/v3/users/1fd2f791559f47b8b99e9bec74fe3b1b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:48.481744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd2f791559f47b8b99e9bec74fe3b1b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:48.485545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:48.486847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f0570-8f37-4b75-8255-270adb6fbdb5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:48.499437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:48.500446 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:48.500789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:48.501155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] PATH_INFO: `/v3/users/7cedf32edfe24fd9b79c2fd783ef4d8e` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:48.503076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cedf32edfe24fd9b79c2fd783ef4d8e)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:48.506941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:48.895998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cedf32edfe24fd9b79c2fd783ef4d8e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.897966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cedf32edfe24fd9b79c2fd783ef4d8e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.934030 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd2f791559f47b8b99e9bec74fe3b1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.937494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cedf32edfe24fd9b79c2fd783ef4d8e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.938026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd2f791559f47b8b99e9bec74fe3b1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.951378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cedf32edfe24fd9b79c2fd783ef4d8e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:48.952440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd2f791559f47b8b99e9bec74fe3b1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:49.022625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd2f791559f47b8b99e9bec74fe3b1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:49.044045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd2f791559f47b8b99e9bec74fe3b1b'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:49.053019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:49.063077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cedf32edfe24fd9b79c2fd783ef4d8e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:49.071909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:49.103101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 885/1778] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:48 2022] DELETE /identity/v3/users/7cedf32edfe24fd9b79c2fd783ef4d8e => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:49.106324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:49.113082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:49.181147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 894/1779] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:48 2022] DELETE /identity/v3/users/1fd2f791559f47b8b99e9bec74fe3b1b => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:49.181147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:49.183336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:49.361184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be7c82-d163-45c6-841b-6f6e7b8d67a9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:49.431804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba93feb-b349-4bd7-b92b-1969fd42fc9b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:49.452799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04c39e-a3a8-43c2-8339-98d757176b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:49.454138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04c39e-a3a8-43c2-8339-98d757176b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:49.454521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04c39e-a3a8-43c2-8339-98d757176b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:49.454732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04c39e-a3a8-43c2-8339-98d757176b0e admin admin] PATH_INFO: `/v3/projects/82a7a4b18d9a40138470bd6ddad6b6d5/users/14ff4d1463194fdcba1ddf179503bed4/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:49.507942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04c39e-a3a8-43c2-8339-98d757176b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82a7a4b18d9a40138470bd6ddad6b6d5, user_id=14ff4d1463194fdcba1ddf179503bed4, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:49.511819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04c39e-a3a8-43c2-8339-98d757176b0e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:49.596530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:49.600189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:49.600189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:49.600189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] PATH_INFO: `/v3/projects/9d81aa7990454c5498824d4bfeac2f46/users/94428745298c4554af22d038dbca1c9e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:49.657477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d81aa7990454c5498824d4bfeac2f46, user_id=94428745298c4554af22d038dbca1c9e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:49.657477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:49.957840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 895/1780] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:49 2022] PUT /identity/v3/projects/82a7a4b18d9a40138470bd6ddad6b6d5/users/14ff4d1463194fdcba1ddf179503bed4/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:49.970618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d65bfb-9690-4242-ad21-6d855f4d5ca9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:49.970916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d65bfb-9690-4242-ad21-6d855f4d5ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:49.971206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d65bfb-9690-4242-ad21-6d855f4d5ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.015360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 886/1781] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:49 2022] PUT /identity/v3/projects/9d81aa7990454c5498824d4bfeac2f46/users/94428745298c4554af22d038dbca1c9e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:50.024477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:50.109793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f5d65bfb-9690-4242-ad21-6d855f4d5ca9 None None] MFA Rules not processed for user `14ff4d1463194fdcba1ddf179503bed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:50.111390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f5d65bfb-9690-4242-ad21-6d855f4d5ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:50.195126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbabb4-c08a-4b21-b341-4ecf8e5ad51d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:50.214122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:50.215321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.215906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.217188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.218386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:50.222669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:50.243882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 887/1782] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:50.249694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:50.289657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d477e-33b4-4c3e-b9a6-723d07c409f2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:50.296715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:50.299959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.300347 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.307077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] PATH_INFO: `/v3/projects/9d81aa7990454c5498824d4bfeac2f46/users/94428745298c4554af22d038dbca1c9e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.314683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d81aa7990454c5498824d4bfeac2f46, user_id=94428745298c4554af22d038dbca1c9e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:50.321998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:50.365195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 888/1783] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:50 2022] PUT /identity/v3/projects/9d81aa7990454c5498824d4bfeac2f46/users/94428745298c4554af22d038dbca1c9e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:50.365370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 896/1784] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:50.366909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:50.371286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.375376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.375676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.375942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:50.481518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] MFA Rules not processed for user `94428745298c4554af22d038dbca1c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:50.520698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:50.536468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89e69c-2e23-42dd-a28d-ed912c09c089 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:50.551560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd7732-1eb1-4b7c-98a6-ff5beca3e241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:50.555350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd7732-1eb1-4b7c-98a6-ff5beca3e241 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.555762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd7732-1eb1-4b7c-98a6-ff5beca3e241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.556236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd7732-1eb1-4b7c-98a6-ff5beca3e241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.557194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd7732-1eb1-4b7c-98a6-ff5beca3e241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:50.559344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd7732-1eb1-4b7c-98a6-ff5beca3e241 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:50.688191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 889/1785] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:50 2022] POST /identity/v3/projects => generated 365 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:50.692127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.695218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.695771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.702337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:50.712300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:50.745441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] MFA Rules not processed for user `3cfb66d6bc254e549d6d355c2bbb440a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:50.750420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:50.761893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 897/1786] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:50.762899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:50.791636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0ac0d-73b3-4dff-a6cc-c5f6e9917b29 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:50.832827 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 890/1787] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:50.833937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:50.836709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:50.898675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:50.899301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.899551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.899924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.900556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:50.902536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:50.957354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e975fc-f2af-407c-9637-f8dc2cfe031b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:50.964819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:50.965756 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:50.966054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:50.967691 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:50.986745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 898/1788] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:50 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:50.988028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:50.990180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.005233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407a428-2326-42e0-8fe4-b9ae97dbd425 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.011873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.012950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.014143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.014662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] PATH_INFO: `/v3/projects/f312fdd45e2f482c88b92a898ea956d3` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.029748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f312fdd45e2f482c88b92a898ea956d3)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.036393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.038498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.040035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.051402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 891/1789] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:51.052529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:51.064614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8adf2-2b06-45d6-8527-98dfff53fc1d service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.070549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.071417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.071850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.072277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.073224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.075014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.090628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 892/1790] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:51.091817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:51.103791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05885020-a7f5-4511-8f0e-20d88a85a858 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.109810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.110722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.111016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.111476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] PATH_INFO: `/v3/projects/7591569d17da461885ed0fa96939e0ef/users/258f54b91270489fb198bbc8278fd8cf/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.139328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7591569d17da461885ed0fa96939e0ef, user_id=258f54b91270489fb198bbc8278fd8cf, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.142362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.238229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 893/1791] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:51 2022] PUT /identity/v3/projects/7591569d17da461885ed0fa96939e0ef/users/258f54b91270489fb198bbc8278fd8cf/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:51.239721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:51.241618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.277528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f312fdd45e2f482c88b92a898ea956d3', 'request_id': 'req-44cc06d5-0717-48a9-a367-45d4fde1d238'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.354946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dec23-407f-448b-8c7c-983c15729a59 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.407830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f312fdd45e2f482c88b92a898ea956d3', 'request_id': 'req-44cc06d5-0717-48a9-a367-45d4fde1d238'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.415875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f312fdd45e2f482c88b92a898ea956d3', 'request_id': 'req-44cc06d5-0717-48a9-a367-45d4fde1d238'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.430704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] The token cache is being invalidate because project f312fdd45e2f482c88b92a898ea956d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:51.431182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.432039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.434910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 899/1792] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:50 2022] DELETE /identity/v3/projects/f312fdd45e2f482c88b92a898ea956d3 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:51.436130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:51.437833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.472487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.473732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.474162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.474583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] PATH_INFO: `/v3/projects/f66bd6ae98a14ff7bbe3c703a0c4cbb6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.479057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66bd6ae98a14ff7bbe3c703a0c4cbb6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.492410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.558920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc06d5-0717-48a9-a367-45d4fde1d238 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.565197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.565974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.566332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.566782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.567491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.570264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.581919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 900/1793] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:51.583696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:51.587787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.647217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66bd6ae98a14ff7bbe3c703a0c4cbb6', 'request_id': 'req-67cdcc43-9085-459c-b427-42c438274019'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.661901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66bd6ae98a14ff7bbe3c703a0c4cbb6', 'request_id': 'req-67cdcc43-9085-459c-b427-42c438274019'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.681010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451ccb3-b002-421b-a1b3-9b62c73688b0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.741803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66bd6ae98a14ff7bbe3c703a0c4cbb6', 'request_id': 'req-67cdcc43-9085-459c-b427-42c438274019'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.776775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] The token cache is being invalidate because project f66bd6ae98a14ff7bbe3c703a0c4cbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:51.777125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.777844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67cdcc43-9085-459c-b427-42c438274019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:51.778245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.779294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.780175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.780762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.783334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.787848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 894/1794] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:51 2022] DELETE /identity/v3/projects/f66bd6ae98a14ff7bbe3c703a0c4cbb6 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:51.789657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4ef63-9f69-417d-ba5d-b2dea764386a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.789954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4ef63-9f69-417d-ba5d-b2dea764386a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.790234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4ef63-9f69-417d-ba5d-b2dea764386a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.821496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-91d4ef63-9f69-417d-ba5d-b2dea764386a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:51.823993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:51.824863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-91d4ef63-9f69-417d-ba5d-b2dea764386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.873774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.879346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.895110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 901/1795] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:51.905754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:51.907401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.909678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 895/1796] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:51.912953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee426c0-33ca-4609-adcf-c5c12f923e7f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.913296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee426c0-33ca-4609-adcf-c5c12f923e7f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.913577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee426c0-33ca-4609-adcf-c5c12f923e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.937486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6ee426c0-33ca-4609-adcf-c5c12f923e7f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:51.938608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6ee426c0-33ca-4609-adcf-c5c12f923e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:51.953025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 896/1797] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:51.954676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4336-eb63-4093-9b42-8ee548e5fa38 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.954970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4336-eb63-4093-9b42-8ee548e5fa38 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.955250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4336-eb63-4093-9b42-8ee548e5fa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.960233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b9c14-8733-42b8-badc-a50fe5fd3778 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:51.966159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df6fe0-7b89-42be-a8ee-8f584a56c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:51.968040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df6fe0-7b89-42be-a8ee-8f584a56c7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:51.968567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df6fe0-7b89-42be-a8ee-8f584a56c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:51.969010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df6fe0-7b89-42be-a8ee-8f584a56c7b7 admin admin] PATH_INFO: `/v3/projects/7591569d17da461885ed0fa96939e0ef/users/258f54b91270489fb198bbc8278fd8cf/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:51.971687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df6fe0-7b89-42be-a8ee-8f584a56c7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7591569d17da461885ed0fa96939e0ef, user_id=258f54b91270489fb198bbc8278fd8cf, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:51.973717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:51.977829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df6fe0-7b89-42be-a8ee-8f584a56c7b7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:51.979618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f40a4336-eb63-4093-9b42-8ee548e5fa38 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:51.983033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f40a4336-eb63-4093-9b42-8ee548e5fa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:52.002227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 902/1798] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:51 2022] PUT /identity/v3/projects/7591569d17da461885ed0fa96939e0ef/users/258f54b91270489fb198bbc8278fd8cf/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:52.003449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.003748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.004032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.027551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:52.029038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:52.106686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 897/1799] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:52.109488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.112299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.113461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.140669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 903/1800] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:52.140669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.144121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:52.151430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd0a1-76ef-483b-b573-97570147f9a9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.156081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.157018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.157127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] MFA Rules not processed for user `258f54b91270489fb198bbc8278fd8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:52.158228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:52.158528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.158827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.159545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:52.159852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.161475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.175189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 904/1801] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:52.178023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.197554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7a7f0-e7dd-4229-8cd7-b796b520aae3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.205302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.205993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.206280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.206581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.207298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:52.207604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.212998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.244301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 905/1802] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:52.245401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.271904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba819d-1917-4b11-ad72-e91731795c63 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.278459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952187-6356-4333-8291-9941825092c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.278641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952187-6356-4333-8291-9941825092c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.279469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952187-6356-4333-8291-9941825092c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.279898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952187-6356-4333-8291-9941825092c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.285399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17952187-6356-4333-8291-9941825092c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.285555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 898/1803] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:52.289995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.297686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17952187-6356-4333-8291-9941825092c7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.308490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55488c94-243a-4114-9822-af8f0f25da8c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.318142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.319273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.344487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.344487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.349057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.355398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.449457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 906/1804] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:52 2022] POST /identity/v3/projects => generated 399 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:52.451311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952187-6356-4333-8291-9941825092c7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.454975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-17952187-6356-4333-8291-9941825092c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:52.495093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 899/1805] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:52 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:52.495941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.549438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952187-6356-4333-8291-9941825092c7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.635364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc8bc2-2f85-4e8e-849c-d7ab95afe170 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.637656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.637824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.638925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.639089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.639261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.639425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.639534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.639768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.640643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.642632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.721190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.723571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.738489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 907/1806] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:52.739221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.758938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 900/1807] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:52 2022] POST /identity/v3/users => generated 300 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:52.759996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.777767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f5b70-e409-497c-a7e9-25183bfbeecd service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.781971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808e94e-a890-4379-89c5-47ef3ab74fd9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.782902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.783620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.783865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.784144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.784984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.793489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.793489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.793679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.793769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.793769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.793769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.793769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.803111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 901/1808] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:52.812944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.823099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0569f-6f80-422e-963e-39f140fe0950 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.827975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.828749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.829038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.829313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.830059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.831763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.843946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 902/1809] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:49:52.848090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.859073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f7faf-8638-43ee-b9c0-35d9860910f2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.864246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7f20c-ce83-4c62-b4c1-ccbca21890e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.865170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7f20c-ce83-4c62-b4c1-ccbca21890e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.865452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7f20c-ce83-4c62-b4c1-ccbca21890e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.866104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7f20c-ce83-4c62-b4c1-ccbca21890e9 admin admin] PATH_INFO: `/v3/projects/825497fb84f44b45a190b91d60b04e54/users/661fd03a1e264f80b892eb34ad2601e0/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.882086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7f20c-ce83-4c62-b4c1-ccbca21890e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=825497fb84f44b45a190b91d60b04e54, user_id=661fd03a1e264f80b892eb34ad2601e0, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.885015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7f20c-ce83-4c62-b4c1-ccbca21890e9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.915316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:52.928435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 908/1810] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:52 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:52.936239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:52.949816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624e29c-bde5-497c-85e8-d7247c442884 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:52.957837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:52.959266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:52.959949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:52.960243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:52.961287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:52.964764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:52.987388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 909/1811] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:52.988696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.000416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074be8b7-8790-4801-bdec-982121ed3ebd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.008230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 903/1812] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:52 2022] PUT /identity/v3/projects/825497fb84f44b45a190b91d60b04e54/users/661fd03a1e264f80b892eb34ad2601e0/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:53.019646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.019646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.019646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.052907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] MFA Rules not processed for user `661fd03a1e264f80b892eb34ad2601e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:53.055479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:53.080248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.080965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.081443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.081875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.082778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.085963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.105440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 910/1813] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:53.110802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.130511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cfed-dc22-4a28-ad73-bba6232b3f70 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.140901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633fd91-1e39-4915-bda2-0fd456d0d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.142789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633fd91-1e39-4915-bda2-0fd456d0d5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.143580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633fd91-1e39-4915-bda2-0fd456d0d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.144266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633fd91-1e39-4915-bda2-0fd456d0d5a4 admin admin] PATH_INFO: `/v3/projects/736b920fd8304e34b56fdee59a555f14/users/73bc6db96e7b4f73916a7cb66df57e65/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.162439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 904/1814] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:53.167988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.179822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633fd91-1e39-4915-bda2-0fd456d0d5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736b920fd8304e34b56fdee59a555f14, user_id=73bc6db96e7b4f73916a7cb66df57e65, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.188244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633fd91-1e39-4915-bda2-0fd456d0d5a4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.239893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954d5c2-1a41-4038-8d86-256669edfd09 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.243651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.244414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.244665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.244944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.246938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:53.308507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.310032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.319496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 905/1815] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:53.320309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.322938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:53.328154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 911/1816] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:53 2022] PUT /identity/v3/projects/736b920fd8304e34b56fdee59a555f14/users/73bc6db96e7b4f73916a7cb66df57e65/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:53.338932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.339232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.339500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.345076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438c15d-d317-41bd-bc62-30c30f3b270d service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.428349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.429548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.430715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.431291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.432330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.444652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.451333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] MFA Rules not processed for user `73bc6db96e7b4f73916a7cb66df57e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:53.452647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:53.471451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:53.577529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 906/1817] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:53 2022] POST /identity/v3/projects => generated 401 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:49:53.581318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.589001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 912/1818] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:53.594417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.626924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efac67-2a1d-4743-9eed-0fe5eedc2b43 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.671249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4207ac9-3ba5-4f99-8d2a-235c6622e44c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.678299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.678299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.678299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.678299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.678924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.681181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.684386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.685278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.686225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.687374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.688221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.690964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.783599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 907/1819] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:53 2022] POST /identity/v3/users => generated 307 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:49:53.784988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 913/1820] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:53 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:49:53.787807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.789882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.812052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a313768-9527-43c0-8f0d-f44a621e93a5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.905376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444794-ecf4-4e0e-85af-777f929b89d2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.911931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.913023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.913298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.913690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.914601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.916872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.928476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.929304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.931426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:53.931543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.931543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.931543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.932315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.943063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 908/1821] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:53.947155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:53.957664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a8f35-7fe3-49cc-98b8-8766eb59970d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:53.964641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:53.971860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:53.971860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:53.971860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] PATH_INFO: `/v3/projects/70040405a6dd4bc59c80f05110057581/users/73ea0de53e1d464ca7e4a09e5e8cb915/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:53.983683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70040405a6dd4bc59c80f05110057581, user_id=73ea0de53e1d464ca7e4a09e5e8cb915, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:53.987044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:53.990503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:54.044115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 914/1822] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:49:53 2022] POST /identity/v3/users => generated 296 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:49:54.046538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.057254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14799fde-9785-4528-bf42-9d755f899a8c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.062396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.069561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.069561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.069561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.069561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.069561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.072784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 909/1823] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:53 2022] PUT /identity/v3/projects/70040405a6dd4bc59c80f05110057581/users/73ea0de53e1d464ca7e4a09e5e8cb915/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:54.073828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.079805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 915/1824] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:49:54.080830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.091036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449180c-b4cc-4291-860d-299fb531e707 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.140414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66d5aa-4bc7-4fb9-a66a-fae3645a75e9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.149565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.150805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.152034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.152156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.153042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.155521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.166872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.167897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.168254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.168624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 910/1825] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:54.171147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] PATH_INFO: `/v3/projects/4614fdb2e7c44211b9e6026d05de2f3b/users/31b9e4172fbe467f9cf9f26e8220deca/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.175937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.186976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7e243-f02c-4d50-b847-5bce7f992dc3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.190710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4614fdb2e7c44211b9e6026d05de2f3b, user_id=31b9e4172fbe467f9cf9f26e8220deca, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.192573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc825b-f62b-4b68-b200-85df6824f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.193642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc825b-f62b-4b68-b200-85df6824f184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.193847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc825b-f62b-4b68-b200-85df6824f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.194862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc825b-f62b-4b68-b200-85df6824f184 admin admin] PATH_INFO: `/v3/projects/70040405a6dd4bc59c80f05110057581/users/73ea0de53e1d464ca7e4a09e5e8cb915/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.194961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.196363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc825b-f62b-4b68-b200-85df6824f184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70040405a6dd4bc59c80f05110057581, user_id=73ea0de53e1d464ca7e4a09e5e8cb915, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.199879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc825b-f62b-4b68-b200-85df6824f184 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.225909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 911/1826] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:54 2022] PUT /identity/v3/projects/70040405a6dd4bc59c80f05110057581/users/73ea0de53e1d464ca7e4a09e5e8cb915/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:54.237913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.238208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.238942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.267999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 916/1827] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:54 2022] PUT /identity/v3/projects/4614fdb2e7c44211b9e6026d05de2f3b/users/31b9e4172fbe467f9cf9f26e8220deca/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:54.269028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.269618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] MFA Rules not processed for user `73ea0de53e1d464ca7e4a09e5e8cb915`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:54.270862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:54.337600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f189c-949d-4790-a4e1-5c3686a263cb admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.344800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.345673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.350131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.350131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.350131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.350131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.355735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 912/1828] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:54.363284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 917/1829] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:49:54.368618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.370724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.379733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:54.383664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924035-1593-4c8c-bb62-9195793f76a9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.389024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca0db7-3645-46e1-a00a-f31f9d12898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.389941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0db7-3645-46e1-a00a-f31f9d12898c admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.390229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0db7-3645-46e1-a00a-f31f9d12898c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.390530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca0db7-3645-46e1-a00a-f31f9d12898c admin admin] PATH_INFO: `/v3/projects/4614fdb2e7c44211b9e6026d05de2f3b/users/31b9e4172fbe467f9cf9f26e8220deca/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.392611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0db7-3645-46e1-a00a-f31f9d12898c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4614fdb2e7c44211b9e6026d05de2f3b, user_id=31b9e4172fbe467f9cf9f26e8220deca, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.395324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca0db7-3645-46e1-a00a-f31f9d12898c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.408190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 918/1830] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:49:54 2022] PUT /identity/v3/projects/4614fdb2e7c44211b9e6026d05de2f3b/users/31b9e4172fbe467f9cf9f26e8220deca/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:49:54.412750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:54.421064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.421365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.421650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.427516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458672fd-6db1-4933-8756-a56fe386d890 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.449894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] MFA Rules not processed for user `31b9e4172fbe467f9cf9f26e8220deca`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:54.451323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:54.480874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.481711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.482692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.483013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.555884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 919/1831] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:49:54.568028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.569891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.581938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.583578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:54.587252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 913/1832] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:54 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:54.604203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e146-541f-4a75-abb5-a1067b5ecfb0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.609151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.609819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.613288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.613288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.620292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.621783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.632748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 920/1833] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:54.759889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:54.811934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:54.820732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eebbc1-7461-471d-a7db-b84a4c0c49fc service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:54.824621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:54.825286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:54.825571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:54.827262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:54.892097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:54.894101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:54.895634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:54.909549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 914/1834] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:49:57.717887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:57.735511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbe5cc-a5ab-4f3c-b39e-b99f762026f1 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:57.740003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b9e4172fbe467f9cf9f26e8220deca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4614fdb2e7c44211b9e6026d05de2f3b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:57.741509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:57.741805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:57.743005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:57.745202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:57.746011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:57.747912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:57.768604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 921/1835] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:49:57.773066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:57.792737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a426f9e-3a6d-41b1-9ed5-e838ebba0baf service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:57.797727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b9e4172fbe467f9cf9f26e8220deca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4614fdb2e7c44211b9e6026d05de2f3b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:57.798469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:57.798721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:57.802313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:57.802313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:57.802313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:57.911726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 915/1836] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:49:57 2022] POST /identity/v3/projects => generated 353 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:49:57.915641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:57.927165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da225525-e108-47dc-9d00-01336d8f5e7e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:57.997155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b9e4172fbe467f9cf9f26e8220deca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4614fdb2e7c44211b9e6026d05de2f3b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:57.998945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:57.999107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:57.999409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:58.000149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:49:58.000463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:58.002085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:58.014392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 922/1837] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:49:58.057252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:58.068390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6ca7b-0199-49ac-b00e-888bb0ff564d tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:58.073037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b9e4172fbe467f9cf9f26e8220deca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4614fdb2e7c44211b9e6026d05de2f3b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:58.073845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:58.074132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:58.074408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] PATH_INFO: `/v3/projects/cebbc583ca26482c8472034274bee303` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:58.075817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cebbc583ca26482c8472034274bee303)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:58.077770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:58.173852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebbc583ca26482c8472034274bee303', 'request_id': 'req-3aeb4858-c0f9-4306-b292-faf635077e1a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:58.179819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebbc583ca26482c8472034274bee303', 'request_id': 'req-3aeb4858-c0f9-4306-b292-faf635077e1a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:58.246013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebbc583ca26482c8472034274bee303', 'request_id': 'req-3aeb4858-c0f9-4306-b292-faf635077e1a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:58.259510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] The token cache is being invalidate because project cebbc583ca26482c8472034274bee303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:49:58.261902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:58.261902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3aeb4858-c0f9-4306-b292-faf635077e1a tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:58.262363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 916/1838] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:49:58 2022] DELETE /identity/v3/projects/cebbc583ca26482c8472034274bee303 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:58.893919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:58.896085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:58.918472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:59.000186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf5eb2-1feb-4a90-a02c-e798b16c5d1e tempest-VolumeQuotaClassesTest-707527210 tempest-VolumeQuotaClassesTest-707527210-project-admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:59.075877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:59.077537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:59.077824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:59.078098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] PATH_INFO: `/v3/users/73bc6db96e7b4f73916a7cb66df57e65` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:59.083938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73bc6db96e7b4f73916a7cb66df57e65)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:59.083938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:59.369101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73bc6db96e7b4f73916a7cb66df57e65'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:59.369686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73bc6db96e7b4f73916a7cb66df57e65'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:59.370306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73bc6db96e7b4f73916a7cb66df57e65'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:59.382442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73bc6db96e7b4f73916a7cb66df57e65'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:59.389869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73bc6db96e7b4f73916a7cb66df57e65'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:49:59.464051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 923/1839] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:49:58 2022] DELETE /identity/v3/users/73bc6db96e7b4f73916a7cb66df57e65 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:49:59.469820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:59.470289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:59.470556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:59.488891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:49:59.489598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:59.552429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 917/1840] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:49:59.561891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:49:59.563313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:49:59.609202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:59.624374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b8aa0-5792-4292-ad3d-f7506be0525f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:49:59.628200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:49:59.629213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:49:59.629884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:49:59.630293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:49:59.640484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:49:59.641953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:49:59.652242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 924/1841] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:00.090409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:00.091729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:00.109919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb8f93-2077-473b-a1de-2c041f647a08 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:00.115923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:00.124393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:00.137769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:00.138177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] PATH_INFO: `/v3/projects/736b920fd8304e34b56fdee59a555f14` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:00.140304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=736b920fd8304e34b56fdee59a555f14)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:00.142365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:00.176868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:00.329900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736b920fd8304e34b56fdee59a555f14', 'request_id': 'req-0a22206d-d868-44b1-a78e-c4581ef248dc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.336596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736b920fd8304e34b56fdee59a555f14', 'request_id': 'req-0a22206d-d868-44b1-a78e-c4581ef248dc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.406845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736b920fd8304e34b56fdee59a555f14', 'request_id': 'req-0a22206d-d868-44b1-a78e-c4581ef248dc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.421138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] The token cache is being invalidate because project 736b920fd8304e34b56fdee59a555f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:00.421358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.422275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a22206d-d868-44b1-a78e-c4581ef248dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.424074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 918/1842] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:00 2022] DELETE /identity/v3/projects/736b920fd8304e34b56fdee59a555f14 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:00.435042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:00.439302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0383f-defd-405f-81d8-8233afc5ce00 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:00.504647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:00.505506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:00.505797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:00.506078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] PATH_INFO: `/v3/users/31b9e4172fbe467f9cf9f26e8220deca` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:00.507287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b9e4172fbe467f9cf9f26e8220deca)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:00.509352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:00.820797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b9e4172fbe467f9cf9f26e8220deca'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.821485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b9e4172fbe467f9cf9f26e8220deca'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.822096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b9e4172fbe467f9cf9f26e8220deca'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.836935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b9e4172fbe467f9cf9f26e8220deca'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.845741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dd66a6f0-e1b6-45c7-8f27-53984398304a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b9e4172fbe467f9cf9f26e8220deca'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:00.960708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 925/1843] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:00 2022] DELETE /identity/v3/users/31b9e4172fbe467f9cf9f26e8220deca => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:01.349866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c89b-9416-49b6-974b-a8f19ec94e9b None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:01.352660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c89b-9416-49b6-974b-a8f19ec94e9b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:01.352660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c89b-9416-49b6-974b-a8f19ec94e9b None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:01.353177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 919/1844] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:50:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:50:01.369021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:01.369314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:01.369655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:01.382621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5c89b-9416-49b6-974b-a8f19ec94e9b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:01.384999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9cb5c89b-9416-49b6-974b-a8f19ec94e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:01.392826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:01.394986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:01.507525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:01.517209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5c89b-9416-49b6-974b-a8f19ec94e9b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:01.526655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:01.527553 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:01.527842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:01.528124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] PATH_INFO: `/v3/projects/4614fdb2e7c44211b9e6026d05de2f3b` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:01.531264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4614fdb2e7c44211b9e6026d05de2f3b)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:01.533681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:01.557080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 926/1845] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:01.675349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4614fdb2e7c44211b9e6026d05de2f3b', 'request_id': 'req-347a7e0a-d403-4c34-b86e-9b0d7269ff34'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:01.675349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4614fdb2e7c44211b9e6026d05de2f3b', 'request_id': 'req-347a7e0a-d403-4c34-b86e-9b0d7269ff34'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:01.725892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:01.727446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:01.742662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483acb41-e66c-48d9-a9aa-e1692561e1b1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:01.806658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bab7b5-5016-4394-a2b1-b253909e0195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:01.807308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bab7b5-5016-4394-a2b1-b253909e0195 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:01.807595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bab7b5-5016-4394-a2b1-b253909e0195 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:01.807872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bab7b5-5016-4394-a2b1-b253909e0195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:01.817462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bab7b5-5016-4394-a2b1-b253909e0195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:01.819047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bab7b5-5016-4394-a2b1-b253909e0195 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:01.829493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 927/1846] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:01.851734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4614fdb2e7c44211b9e6026d05de2f3b', 'request_id': 'req-347a7e0a-d403-4c34-b86e-9b0d7269ff34'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:01.864748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] The token cache is being invalidate because project 4614fdb2e7c44211b9e6026d05de2f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:01.865006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:01.865644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-347a7e0a-d403-4c34-b86e-9b0d7269ff34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:01.867305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 920/1847] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:01 2022] DELETE /identity/v3/projects/4614fdb2e7c44211b9e6026d05de2f3b => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:01.872323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:01.872643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:01.873072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:01.892600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:01.893808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:01.961725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 928/1848] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:01.971164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:01.971660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:01.972009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:01.992365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:01.993393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:02.001890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 921/1849] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:02.006383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.017230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecba1b-7525-4f9c-8490-d6bcc8a01f31 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.022204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.022756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.023045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.023326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.024063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:02.024379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.026598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.040260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 929/1850] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:50:02.044763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.054245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5922a-a3eb-4110-bbb0-7a2b45164b19 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.058868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.059560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.059872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.060173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.061657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.063701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.145764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 922/1851] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:02 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:02.150246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.160806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8a88d-3ff4-437e-8677-830a5b88c68e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.219389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.219980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.220266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.220573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.221328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.223242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.232128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:02.343658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 930/1852] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:02 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:02.347929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.359007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f049598-5661-4916-a0fc-8fde31d874e2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.363701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.364518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.364807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.365081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.365800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.367390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.378533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 923/1853] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:02.382612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.393053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9560a29-62af-4862-ab39-fa7d08c5cc12 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.397657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3196ae-be23-4a83-9bfc-bada8269bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.398311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3196ae-be23-4a83-9bfc-bada8269bed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.398602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3196ae-be23-4a83-9bfc-bada8269bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.398886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3196ae-be23-4a83-9bfc-bada8269bed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.399611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3196ae-be23-4a83-9bfc-bada8269bed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.401260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3196ae-be23-4a83-9bfc-bada8269bed0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.412226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 931/1854] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:02.416584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.422317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:02.427497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7db71-4efa-4009-8110-c648568e2eec admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.431965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.432907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.433190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.433471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] PATH_INFO: `/v3/projects/8ca579f7a7d84e839d3be05c14c230d6/users/42ff664ffa51421fb54f47aae11d5d33/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.449331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca579f7a7d84e839d3be05c14c230d6, user_id=42ff664ffa51421fb54f47aae11d5d33, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.452288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.520722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 924/1855] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:02 2022] PUT /identity/v3/projects/8ca579f7a7d84e839d3be05c14c230d6/users/42ff664ffa51421fb54f47aae11d5d33/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:02.538763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0985-020f-4497-9396-f905600f702f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.539295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0985-020f-4497-9396-f905600f702f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.539387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0985-020f-4497-9396-f905600f702f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.568547 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d57d0985-020f-4497-9396-f905600f702f None None] MFA Rules not processed for user `42ff664ffa51421fb54f47aae11d5d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:02.569771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d57d0985-020f-4497-9396-f905600f702f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:02.646029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 932/1856] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:02.650592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.724413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddbc32-fa65-46ae-8443-9ece83e53dce admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.739867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.741185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.741806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.742157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.743029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.745157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.826937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 925/1857] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:02 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:02.830900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0985-020f-4497-9396-f905600f702f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:02.844085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0985-020f-4497-9396-f905600f702f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:02.910711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:02.911519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:02.912043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:02.912410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:02.913389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:02.915880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:02.990441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 933/1858] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:02 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:02.994441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:03.005078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da359f14-5e2b-4523-9f48-20e66f08ea30 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:03.014139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:03.014139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.014139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.014139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.014495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:03.014495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:03.025190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 926/1859] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:03.029393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:03.040647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987447df-b53a-4a61-b0c4-01a25877d5e4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:03.046158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:03.047047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.047378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.047651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] PATH_INFO: `/v3/projects/48bdc919aaf74f41b1c1d1a7fb276b5e/users/1b9c8664bf5c47d29abf97bb2ed100cd/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.063227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bdc919aaf74f41b1c1d1a7fb276b5e, user_id=1b9c8664bf5c47d29abf97bb2ed100cd, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:03.065986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:03.159903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 934/1860] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:03 2022] PUT /identity/v3/projects/48bdc919aaf74f41b1c1d1a7fb276b5e/users/1b9c8664bf5c47d29abf97bb2ed100cd/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:03.164011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:03.175292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89b809-14fb-4057-83ce-707b12d05d77 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:03.240482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e104a-8f0b-4b26-a45d-46e716a9955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:03.241181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e104a-8f0b-4b26-a45d-46e716a9955b admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.241488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e104a-8f0b-4b26-a45d-46e716a9955b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.241762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e104a-8f0b-4b26-a45d-46e716a9955b admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.242482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e104a-8f0b-4b26-a45d-46e716a9955b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:03.244123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e104a-8f0b-4b26-a45d-46e716a9955b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:03.254774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 927/1861] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:03.258989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:03.269907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2ad5-005f-4333-b8b6-7dedb727f9c6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:03.274582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:03.275435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.275713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.275983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] PATH_INFO: `/v3/projects/48bdc919aaf74f41b1c1d1a7fb276b5e/users/1b9c8664bf5c47d29abf97bb2ed100cd/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.277956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bdc919aaf74f41b1c1d1a7fb276b5e, user_id=1b9c8664bf5c47d29abf97bb2ed100cd, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:03.280942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:03.291104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 935/1862] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:03 2022] PUT /identity/v3/projects/48bdc919aaf74f41b1c1d1a7fb276b5e/users/1b9c8664bf5c47d29abf97bb2ed100cd/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:03.301769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.302052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.302326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.329305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] MFA Rules not processed for user `1b9c8664bf5c47d29abf97bb2ed100cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:03.330520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:03.368377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:03.398193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 928/1863] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:03.409266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:03.411115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:03.441838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:03.473778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25942eb3-c414-44af-a166-298f10ee4277 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:03.477490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:03.478138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.478431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.478706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.488351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:03.489853 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:03.502186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 936/1864] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:03.741628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:03.752120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761762a-7f02-4479-a2e4-6296b6b2bf7c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:03.756370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8beb7d6-b5a4-4e19-a892-69bda63adadb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:03.757443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beb7d6-b5a4-4e19-a892-69bda63adadb service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:03.757596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beb7d6-b5a4-4e19-a892-69bda63adadb service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:03.757826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8beb7d6-b5a4-4e19-a892-69bda63adadb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:03.823275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8beb7d6-b5a4-4e19-a892-69bda63adadb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:03.825128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8beb7d6-b5a4-4e19-a892-69bda63adadb service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:03.842954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 929/1865] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:05.058832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:05.067315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:05.200714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e1cf9-02ac-4b3a-8a82-b3ae2698108a service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:05.220827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:05.220827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:05.220827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:05.223425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] PATH_INFO: `/v3/users/661fd03a1e264f80b892eb34ad2601e0` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:05.226589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661fd03a1e264f80b892eb34ad2601e0)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:05.230990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:05.529233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661fd03a1e264f80b892eb34ad2601e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:05.530043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661fd03a1e264f80b892eb34ad2601e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:05.530605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661fd03a1e264f80b892eb34ad2601e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:05.548731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661fd03a1e264f80b892eb34ad2601e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:05.556201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661fd03a1e264f80b892eb34ad2601e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:05.688154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 937/1866] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:05 2022] DELETE /identity/v3/users/661fd03a1e264f80b892eb34ad2601e0 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:05.690163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:05.690620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:05.690732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:05.718443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:05.719906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:05.738264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:05.739906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:05.821615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:05.839131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 930/1867] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:05.852379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:05.852616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053012e-2d76-4a95-90e8-ccf9f99f47f9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:05.857817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:05.859897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:05.860371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:05.860727 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] PATH_INFO: `/v3/users/14ff4d1463194fdcba1ddf179503bed4` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:05.862124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ff4d1463194fdcba1ddf179503bed4)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:05.864313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:05.872891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:05.996064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc0e5c3-dbdf-4eea-836b-9005999ba601 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:06.000412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd3a93-fcd5-4272-b9c1-2b53ddf193fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:06.001099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd3a93-fcd5-4272-b9c1-2b53ddf193fb service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:06.001377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd3a93-fcd5-4272-b9c1-2b53ddf193fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:06.001653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd3a93-fcd5-4272-b9c1-2b53ddf193fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:06.011172 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd3a93-fcd5-4272-b9c1-2b53ddf193fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:06.012648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd3a93-fcd5-4272-b9c1-2b53ddf193fb service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:06.023674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 931/1868] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:06.142208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ff4d1463194fdcba1ddf179503bed4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:06.143563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ff4d1463194fdcba1ddf179503bed4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:06.145129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ff4d1463194fdcba1ddf179503bed4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:06.188129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ff4d1463194fdcba1ddf179503bed4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:06.219024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ff4d1463194fdcba1ddf179503bed4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:06.326193 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 938/1869] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:05 2022] DELETE /identity/v3/users/14ff4d1463194fdcba1ddf179503bed4 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:06.331775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:06.332062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:06.332570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:06.355647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:06.358033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:06.369748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:06.439064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 932/1870] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:06.446484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:06.448048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:06.539461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab97f8-999b-41c8-b823-3d3467bf4b2a admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:06.543823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:06.544721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:06.545156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:06.545643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:06.558718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:06.560577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:06.575298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 939/1871] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:06.842862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:06.845482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:06.860043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c7b6e-f949-4d53-add9-7d64dbbb84c4 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:06.866571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:06.873252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:06.873824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:06.874920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] PATH_INFO: `/v3/projects/825497fb84f44b45a190b91d60b04e54` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:06.882165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=825497fb84f44b45a190b91d60b04e54)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:06.885141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:07.031333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825497fb84f44b45a190b91d60b04e54', 'request_id': 'req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.037362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825497fb84f44b45a190b91d60b04e54', 'request_id': 'req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.160947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825497fb84f44b45a190b91d60b04e54', 'request_id': 'req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.176228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] The token cache is being invalidate because project 825497fb84f44b45a190b91d60b04e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:07.176726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.177440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.179503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 933/1872] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:06 2022] DELETE /identity/v3/projects/825497fb84f44b45a190b91d60b04e54 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:07.183807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:07.196382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004201a1-3ac5-41b5-be33-96a48506c129 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:07.292552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:07.293446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:07.293731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:07.294048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] PATH_INFO: `/v3/users/73ea0de53e1d464ca7e4a09e5e8cb915` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:07.295483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ea0de53e1d464ca7e4a09e5e8cb915)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:07.297845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:07.299597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:07.301241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:07.310896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:07.316057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a2607-0cf3-48c4-960e-cd15f9f99cc0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:07.321189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:07.322262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:07.322659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:07.323241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] PATH_INFO: `/v3/users/42ff664ffa51421fb54f47aae11d5d33` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:07.324703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ff664ffa51421fb54f47aae11d5d33)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:07.326880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:07.519680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ea0de53e1d464ca7e4a09e5e8cb915'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.521337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ea0de53e1d464ca7e4a09e5e8cb915'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.524544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ea0de53e1d464ca7e4a09e5e8cb915'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.544072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ea0de53e1d464ca7e4a09e5e8cb915'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.554337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ea0de53e1d464ca7e4a09e5e8cb915'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.573151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ff664ffa51421fb54f47aae11d5d33'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.574833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ff664ffa51421fb54f47aae11d5d33'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.586714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ff664ffa51421fb54f47aae11d5d33'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.587610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ff664ffa51421fb54f47aae11d5d33'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.666914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 940/1873] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:07 2022] DELETE /identity/v3/users/73ea0de53e1d464ca7e4a09e5e8cb915 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:07.726208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-424bb490-f1ec-46ef-8950-c044a1411638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ff664ffa51421fb54f47aae11d5d33'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:07.770812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 934/1874] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:07 2022] DELETE /identity/v3/users/42ff664ffa51421fb54f47aae11d5d33 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:07.772769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:07.773378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:07.773801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:07.774767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:07.780122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:07.804394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:07.806712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:07.818340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:07.899622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 935/1875] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:07.913252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:07.915835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:07.979680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f8d56-6ed8-4a94-a216-26599c318d01 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:07.997940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:08.001149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:08.001456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:08.003142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] PATH_INFO: `/v3/projects/82a7a4b18d9a40138470bd6ddad6b6d5` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:08.004335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5eb7d-bb2e-4fd6-90cd-546098263831 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:08.006740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82a7a4b18d9a40138470bd6ddad6b6d5)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:08.009428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:08.010907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:08.011449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:08.011952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:08.012232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:08.026426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:08.028174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:08.048113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 936/1876] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:08.187360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82a7a4b18d9a40138470bd6ddad6b6d5', 'request_id': 'req-81273fae-6138-4fa6-8f32-41e0af60e602'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.252045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82a7a4b18d9a40138470bd6ddad6b6d5', 'request_id': 'req-81273fae-6138-4fa6-8f32-41e0af60e602'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.261047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82a7a4b18d9a40138470bd6ddad6b6d5', 'request_id': 'req-81273fae-6138-4fa6-8f32-41e0af60e602'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.286568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] The token cache is being invalidate because project 82a7a4b18d9a40138470bd6ddad6b6d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:08.287004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.288419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.291519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 941/1877] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:07 2022] DELETE /identity/v3/projects/82a7a4b18d9a40138470bd6ddad6b6d5 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:08.295296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:08.298169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:08.323581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f30d3-0683-4406-bf11-aff8c83e17f4 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:08.491656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:08.502563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:08.502846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:08.503127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] PATH_INFO: `/v3/users/258f54b91270489fb198bbc8278fd8cf` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:08.507343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=258f54b91270489fb198bbc8278fd8cf)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:08.509390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:08.550778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:08.558138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:08.594109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81273fae-6138-4fa6-8f32-41e0af60e602 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:08.600604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:08.601502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:08.601950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:08.602245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] PATH_INFO: `/v3/projects/70040405a6dd4bc59c80f05110057581` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:08.607854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70040405a6dd4bc59c80f05110057581)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:08.607854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:08.625348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:08.798921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70040405a6dd4bc59c80f05110057581', 'request_id': 'req-750b4cd9-f669-4593-945f-13ff0c8ef757'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.807126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258f54b91270489fb198bbc8278fd8cf'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.807910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258f54b91270489fb198bbc8278fd8cf'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.818348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258f54b91270489fb198bbc8278fd8cf'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.820348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258f54b91270489fb198bbc8278fd8cf'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.886132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70040405a6dd4bc59c80f05110057581', 'request_id': 'req-750b4cd9-f669-4593-945f-13ff0c8ef757'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.894706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70040405a6dd4bc59c80f05110057581', 'request_id': 'req-750b4cd9-f669-4593-945f-13ff0c8ef757'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.906505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] The token cache is being invalidate because project 70040405a6dd4bc59c80f05110057581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:08.906950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.907683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-750b4cd9-f669-4593-945f-13ff0c8ef757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:08.909545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 942/1878] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:08 2022] DELETE /identity/v3/projects/70040405a6dd4bc59c80f05110057581 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:08.915360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6303a1-03c6-4e97-aaf8-1bfd2313f9e9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:08.915657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6303a1-03c6-4e97-aaf8-1bfd2313f9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:08.915944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6303a1-03c6-4e97-aaf8-1bfd2313f9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:08.935592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-2b6303a1-03c6-4e97-aaf8-1bfd2313f9e9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:08.937353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2b6303a1-03c6-4e97-aaf8-1bfd2313f9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:08.971750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258f54b91270489fb198bbc8278fd8cf'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.003645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 937/1879] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:08 2022] DELETE /identity/v3/users/258f54b91270489fb198bbc8278fd8cf => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:09.003645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.003645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:09.017805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 943/1880] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:09.020624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.020979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.021358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.050881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:09.053950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:09.124257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc3b4a-b6ba-4860-9a6f-b08aabb4a0ad admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.134391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.135275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.135742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.136025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] PATH_INFO: `/v3/projects/8ca579f7a7d84e839d3be05c14c230d6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.137852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca579f7a7d84e839d3be05c14c230d6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.140910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.157307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 944/1881] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:09.163081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.177689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc7b8c-2cd5-440e-a34c-379b8f3bf31f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.182840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.183692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.183991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.184264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.185037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:09.185379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.188008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.207638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:09.208688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 945/1882] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:50:09.210053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.221549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6df721-41f5-4976-98c0-87f21bff13e6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.228080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.228792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.229080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.229810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.231386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.234308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.320943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca579f7a7d84e839d3be05c14c230d6', 'request_id': 'req-37aed819-577d-4920-a88b-2a6bb8366726'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.326732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca579f7a7d84e839d3be05c14c230d6', 'request_id': 'req-37aed819-577d-4920-a88b-2a6bb8366726'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.376308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 946/1883] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:09 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:09.381009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.487617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca579f7a7d84e839d3be05c14c230d6', 'request_id': 'req-37aed819-577d-4920-a88b-2a6bb8366726'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.489190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d6dd-876c-4c8a-af8a-1cfadea22f5b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.496243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.496966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.497257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.497543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.498774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.501079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.509001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] The token cache is being invalidate because project 8ca579f7a7d84e839d3be05c14c230d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:09.509401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.510436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.512597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 938/1884] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:09 2022] DELETE /identity/v3/projects/8ca579f7a7d84e839d3be05c14c230d6 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:09.517644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.519735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:09.626804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 947/1885] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:09 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:09.631141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.633029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:09.640952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aed819-577d-4920-a88b-2a6bb8366726 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.647339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.648411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.648784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.649175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] PATH_INFO: `/v3/users/1b9c8664bf5c47d29abf97bb2ed100cd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.651217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cc830-2ed8-41b2-841e-b062062a53c7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.651597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9c8664bf5c47d29abf97bb2ed100cd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.653909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.658194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.658896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.660335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.660451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.663611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.667013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.691606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 948/1886] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:09.697463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.717134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0199169-4389-4379-b516-c924cc39a5e4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.724079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.725171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.725461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.725987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.726636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.728286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.741993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 949/1887] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:09.746129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:09.758242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a7a24-4341-438f-a5ce-57c6993a7d73 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:09.768938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ac618-f4c6-4305-85e9-dccb4bfb68ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:09.770090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ac618-f4c6-4305-85e9-dccb4bfb68ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.770887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ac618-f4c6-4305-85e9-dccb4bfb68ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.775083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ac618-f4c6-4305-85e9-dccb4bfb68ce admin admin] PATH_INFO: `/v3/projects/4401a3ac9ae24ff291ed2f58f8b8d31d/users/588a9c687722442bb3cb3868c8866158/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.820010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ac618-f4c6-4305-85e9-dccb4bfb68ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4401a3ac9ae24ff291ed2f58f8b8d31d, user_id=588a9c687722442bb3cb3868c8866158, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:09.823573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ac618-f4c6-4305-85e9-dccb4bfb68ce admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:09.923430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9c8664bf5c47d29abf97bb2ed100cd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.925280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9c8664bf5c47d29abf97bb2ed100cd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.929143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 950/1888] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:09 2022] PUT /identity/v3/projects/4401a3ac9ae24ff291ed2f58f8b8d31d/users/588a9c687722442bb3cb3868c8866158/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:09.944419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:09.944823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:09.945101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:09.952702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9c8664bf5c47d29abf97bb2ed100cd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:09.957679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9c8664bf5c47d29abf97bb2ed100cd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.010294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] MFA Rules not processed for user `588a9c687722442bb3cb3868c8866158`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:10.020618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.076420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:10.077882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9c8664bf5c47d29abf97bb2ed100cd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.108964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 939/1889] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:09 2022] DELETE /identity/v3/users/1b9c8664bf5c47d29abf97bb2ed100cd => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:10.116646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:10.116646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.145027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 951/1890] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:10.169342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:10.174956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:10.176306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.216019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4a218-66fb-41d5-ba60-2109902b643b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:10.226221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:10.227757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.228246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.228878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] PATH_INFO: `/v3/projects/7591569d17da461885ed0fa96939e0ef` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.230614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7591569d17da461885ed0fa96939e0ef)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:10.237700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:10.299501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b59feb-ec9e-47b2-b302-bd6e88aee755 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:10.399191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7591569d17da461885ed0fa96939e0ef', 'request_id': 'req-d744ca64-a991-4ae3-9930-b6c04a17a769'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.413023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfee052-ba05-48f1-9b3b-637691f1a7ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:10.413948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfee052-ba05-48f1-9b3b-637691f1a7ad service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.414413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfee052-ba05-48f1-9b3b-637691f1a7ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.416123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7591569d17da461885ed0fa96939e0ef', 'request_id': 'req-d744ca64-a991-4ae3-9930-b6c04a17a769'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.417517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfee052-ba05-48f1-9b3b-637691f1a7ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.508653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7591569d17da461885ed0fa96939e0ef', 'request_id': 'req-d744ca64-a991-4ae3-9930-b6c04a17a769'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.532283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] The token cache is being invalidate because project 7591569d17da461885ed0fa96939e0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:10.532681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.533302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d744ca64-a991-4ae3-9930-b6c04a17a769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:10.536862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 940/1891] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:10 2022] DELETE /identity/v3/projects/7591569d17da461885ed0fa96939e0ef => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:10.539300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbe34d-d8d7-4ff2-8125-63cdb4ac3ba0 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.539949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbe34d-d8d7-4ff2-8125-63cdb4ac3ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.541252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfee052-ba05-48f1-9b3b-637691f1a7ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:10.542766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfee052-ba05-48f1-9b3b-637691f1a7ad service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:10.548004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbe34d-d8d7-4ff2-8125-63cdb4ac3ba0 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.561055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 941/1892] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:50:10 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:50:10.562615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526eb9-e975-49a8-9469-85f511761dbc None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.563115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526eb9-e975-49a8-9469-85f511761dbc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.564114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526eb9-e975-49a8-9469-85f511761dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.578606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 952/1893] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:10.591085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.594687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.594818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.599123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-58526eb9-e975-49a8-9469-85f511761dbc None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:10.600861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-58526eb9-e975-49a8-9469-85f511761dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.622512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:10.623918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.700111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 942/1894] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:10.719788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710be369-07c5-46a6-8086-9297a74f40be None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.720714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710be369-07c5-46a6-8086-9297a74f40be None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.720714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710be369-07c5-46a6-8086-9297a74f40be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.744116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-710be369-07c5-46a6-8086-9297a74f40be None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:10.745410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-710be369-07c5-46a6-8086-9297a74f40be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.760211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 953/1895] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:50:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:10.766592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 943/1896] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:10.767549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:10.777199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710be369-07c5-46a6-8086-9297a74f40be None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:10.781248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-710be369-07c5-46a6-8086-9297a74f40be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:10.782153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bcff5-4e87-4228-852d-c6518837efd0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:10.788089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:10.788724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.789012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.789468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.790252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:10.790536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:10.792210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:10.796128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710be369-07c5-46a6-8086-9297a74f40be None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:10.806381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 954/1897] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:50:10.809648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:10.810514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.811010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.811688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.812257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:10.823952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:10.828437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:10.829528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7732b-2edc-4445-ba3a-649042111b4f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:10.835441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:10.836124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:10.836413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:10.836725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:10.837439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:10.840857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:10.855863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 944/1898] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:10.972925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 955/1899] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:10 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:10.979471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:10.988708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfda10d-3746-46bf-9a11-c9909693a97f service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.089896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.094018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.101932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.102952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.103375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.103828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.104748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.107012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.202167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ebfa3-5523-444e-8fc5-31bbf478d4ab admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.211035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.211962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.212249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.212555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] PATH_INFO: `/v3/projects/48bdc919aaf74f41b1c1d1a7fb276b5e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.213811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bdc919aaf74f41b1c1d1a7fb276b5e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.215792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.228903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 945/1900] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:10 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:11.230037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.234713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.256565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76ed6a-42e4-4370-8a75-5db65c1ca9b2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.262647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.263535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.263826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.264165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] PATH_INFO: `/v3/users/947d7ba6b4cd443fbb6e8733ed2706f1` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.266070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947d7ba6b4cd443fbb6e8733ed2706f1)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.268517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.360174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bdc919aaf74f41b1c1d1a7fb276b5e', 'request_id': 'req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.429410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bdc919aaf74f41b1c1d1a7fb276b5e', 'request_id': 'req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.436589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bdc919aaf74f41b1c1d1a7fb276b5e', 'request_id': 'req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.449068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] The token cache is being invalidate because project 48bdc919aaf74f41b1c1d1a7fb276b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:11.449308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.450989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.453658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 956/1901] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:11 2022] DELETE /identity/v3/projects/48bdc919aaf74f41b1c1d1a7fb276b5e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:11.454118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.456670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.522729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947d7ba6b4cd443fbb6e8733ed2706f1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.523491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947d7ba6b4cd443fbb6e8733ed2706f1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.554146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947d7ba6b4cd443fbb6e8733ed2706f1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.555350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947d7ba6b4cd443fbb6e8733ed2706f1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.565709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f4d29-a914-44c3-95ff-fc57d27bde2a admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.571992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfa097-ca85-4dbf-a6cb-ccd3e411d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.572738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfa097-ca85-4dbf-a6cb-ccd3e411d7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.573027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfa097-ca85-4dbf-a6cb-ccd3e411d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.573309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfa097-ca85-4dbf-a6cb-ccd3e411d7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.574059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfa097-ca85-4dbf-a6cb-ccd3e411d7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.575912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfa097-ca85-4dbf-a6cb-ccd3e411d7a3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.590907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 957/1902] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:11.592578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.593652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.594100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.616648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:11.617883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.626173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 958/1903] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:11.627019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.629794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:11.633007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.633958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-52e66daf-10f7-4306-a015-51775c684226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947d7ba6b4cd443fbb6e8733ed2706f1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:11.647677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33e4f3-52ad-4680-aa51-caab749a5990 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.657700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31076afa-22ff-44fc-9b19-7b70184e1154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.657906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 946/1904] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:11 2022] DELETE /identity/v3/users/947d7ba6b4cd443fbb6e8733ed2706f1 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:11.658035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31076afa-22ff-44fc-9b19-7b70184e1154 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.658035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31076afa-22ff-44fc-9b19-7b70184e1154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.658035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31076afa-22ff-44fc-9b19-7b70184e1154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.658249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.658320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31076afa-22ff-44fc-9b19-7b70184e1154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.658389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.658389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.658524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31076afa-22ff-44fc-9b19-7b70184e1154 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.672553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 959/1905] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:11.672553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.672553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.672553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.680204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:11.681452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.695391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:11.703256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.712308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:11.757308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 947/1906] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:11.758211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.767799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79086da-237a-45f0-b623-d7f75c5f6996 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.768860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 960/1907] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:11.769848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.773499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.774393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.774677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.774954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] PATH_INFO: `/v3/projects/699fbb05d96f4d1d8523432349c584bf/users/f375069615c44a76adade6bfcc3d6ee7/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.779785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a781453-ec73-4acc-9df7-f02964b958db None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.786440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.787122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.787414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.787697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.788441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:11.788803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.790427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.797198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=699fbb05d96f4d1d8523432349c584bf, user_id=f375069615c44a76adade6bfcc3d6ee7, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.800063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.807653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 961/1908] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:50:11.807653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.807653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:11.885419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 948/1909] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:11 2022] PUT /identity/v3/projects/699fbb05d96f4d1d8523432349c584bf/users/f375069615c44a76adade6bfcc3d6ee7/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:11.885419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:11.910832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba2a7-96f4-4360-8051-6f9e0f0433c9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.972250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f212-b2be-4fe9-9456-a886fe40f13b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:11.977147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:11.977826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:11.978116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:11.978398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:11.979097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:11.981080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:11.999054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cb39d-0a4f-4430-b42d-379f3fe326a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.000456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cb39d-0a4f-4430-b42d-379f3fe326a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.000767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cb39d-0a4f-4430-b42d-379f3fe326a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.001048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cb39d-0a4f-4430-b42d-379f3fe326a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.012029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cb39d-0a4f-4430-b42d-379f3fe326a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.014444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cb39d-0a4f-4430-b42d-379f3fe326a5 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.028622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 962/1910] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:12.029975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.030273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.032089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.065384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] MFA Rules not processed for user `f375069615c44a76adade6bfcc3d6ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:12.067344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:12.100574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 949/1911] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:11 2022] POST /identity/v3/projects => generated 381 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:12.122229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.186169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 963/1912] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:12.199575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.201297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:12.252707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e9dd-3953-41e9-afeb-d15c2499556a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:12.260980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.261963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.262256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.262538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.263712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.268205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.270031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb0029-10c3-4c5f-9dc9-10cbe0adbea9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:12.367828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.369470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.369761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.370132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.406880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 950/1913] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:12 2022] POST /identity/v3/users => generated 291 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:12.410927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.421540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cce8b4-b138-44e7-a4af-a1223b0b7e7f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:12.427403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.428811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.429453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.430234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.431297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.433409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.450525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 951/1914] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:50:12.451994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.462816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bc5af-4cf4-40ce-93a2-1e5a5263e7e0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:12.471240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.472305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.472826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.473309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.474271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.476107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.487854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.487854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.494016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 964/1915] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:12.500330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 952/1916] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:12.507135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.520962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.525681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722f26-34cb-457a-94a1-0731c8045578 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:12.531863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d8fdb-45d3-4a1a-ad7b-ad8f5c53d251 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:12.533868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afc750-89df-4f3f-938f-be76c88fadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.537506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afc750-89df-4f3f-938f-be76c88fadca admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.537802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afc750-89df-4f3f-938f-be76c88fadca admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.538090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afc750-89df-4f3f-938f-be76c88fadca admin admin] PATH_INFO: `/v3/projects/edf48691aca94a17907346887ef5fa85/users/e4e868ba0ae7450db85857f0ac84f831/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.540781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:12.541670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.541970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.542232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] PATH_INFO: `/v3/users/f375069615c44a76adade6bfcc3d6ee7` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.543476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f375069615c44a76adade6bfcc3d6ee7)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.545594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.561250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afc750-89df-4f3f-938f-be76c88fadca admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf48691aca94a17907346887ef5fa85, user_id=e4e868ba0ae7450db85857f0ac84f831, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:12.569435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afc750-89df-4f3f-938f-be76c88fadca admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:12.704754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 965/1917] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:12 2022] PUT /identity/v3/projects/edf48691aca94a17907346887ef5fa85/users/e4e868ba0ae7450db85857f0ac84f831/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:12.716894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.717194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.717474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.753168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] MFA Rules not processed for user `e4e868ba0ae7450db85857f0ac84f831`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:12.754915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:12.817464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f375069615c44a76adade6bfcc3d6ee7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:12.819554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f375069615c44a76adade6bfcc3d6ee7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:12.830608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f375069615c44a76adade6bfcc3d6ee7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:12.831734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f375069615c44a76adade6bfcc3d6ee7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:12.864547 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 966/1918] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:12.867178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:12.891353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:12.909015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e0242396-a103-4a2d-b693-25222fc25ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f375069615c44a76adade6bfcc3d6ee7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:12.939616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 953/1919] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:12 2022] DELETE /identity/v3/users/f375069615c44a76adade6bfcc3d6ee7 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:12.944155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:12.944393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:12.946111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:12.988549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:12.989723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:13.007724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c7713-6d47-4d87-a95f-8ad2280e49a2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.074264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 954/1920] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:13.092554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.095025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:13.097197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.097971 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.098327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.098678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.099477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.102262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.144622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:13.192027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6c4ea-0621-4021-91fc-67dff1a438f8 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.198323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.200040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.200326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.200843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.216992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.219431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.232691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 967/1921] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:12 2022] POST /identity/v3/projects => generated 381 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:13.234376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.237247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 955/1922] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:13.243127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.243249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:13.255253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601748f0-c714-47c9-9e20-8762e5e5a9c4 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.378575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad451f70-a098-4863-81c1-577b11ac0666 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.384534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.385437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.385728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.386638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] PATH_INFO: `/v3/projects/759443049094408189e8bf4daebea4ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.387924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=759443049094408189e8bf4daebea4ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.391855 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.463276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.463964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.464253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.464590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.465591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.470395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.610649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 956/1923] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:13 2022] POST /identity/v3/users => generated 297 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:13.612136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.628095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759443049094408189e8bf4daebea4ea', 'request_id': 'req-723dce90-939c-463a-bdef-32a72c81a55f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:13.628461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a8a4a-2ccb-4d21-bc6c-cee3995c6752 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.704145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759443049094408189e8bf4daebea4ea', 'request_id': 'req-723dce90-939c-463a-bdef-32a72c81a55f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:13.712729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759443049094408189e8bf4daebea4ea', 'request_id': 'req-723dce90-939c-463a-bdef-32a72c81a55f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:13.735849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.736572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.736896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.737189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.738171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] The token cache is being invalidate because project 759443049094408189e8bf4daebea4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:13.738460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.739357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:13.740226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:13.741294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.742401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 968/1924] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:13 2022] DELETE /identity/v3/projects/759443049094408189e8bf4daebea4ea => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:13.746489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.748229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:13.759432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 957/1925] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:50:13.763491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.765832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:13.767953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dce90-939c-463a-bdef-32a72c81a55f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.774173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.776109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.776610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.777046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] PATH_INFO: `/v3/users/94428745298c4554af22d038dbca1c9e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.779005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94428745298c4554af22d038dbca1c9e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.780393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06b89-f96a-4bd2-bdd9-db50400ae2cb admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.782252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.787085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.788162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.788660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.789011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] PATH_INFO: `/v3/projects/d64cee7a9e3f48469888895651dd76b4/users/c1849a4b1c324106a9efce7ad1dc0d15/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.807379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64cee7a9e3f48469888895651dd76b4, user_id=c1849a4b1c324106a9efce7ad1dc0d15, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.810577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:13.895936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 958/1926] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:13 2022] PUT /identity/v3/projects/d64cee7a9e3f48469888895651dd76b4/users/c1849a4b1c324106a9efce7ad1dc0d15/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:13.902805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:13.902805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:13.976184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717892d2-2629-4cd6-9e1a-d91e076298ba admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:13.982179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:13.983176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:13.984226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:13.984957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] PATH_INFO: `/v3/projects/699fbb05d96f4d1d8523432349c584bf` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:13.986388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699fbb05d96f4d1d8523432349c584bf)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:13.988760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.002585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:14.078792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94428745298c4554af22d038dbca1c9e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.079466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94428745298c4554af22d038dbca1c9e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.080059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94428745298c4554af22d038dbca1c9e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.099199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94428745298c4554af22d038dbca1c9e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.104648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699fbb05d96f4d1d8523432349c584bf', 'request_id': 'req-47365774-fe6c-4f10-86fa-ee45194cf368'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.107785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94428745298c4554af22d038dbca1c9e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.113459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699fbb05d96f4d1d8523432349c584bf', 'request_id': 'req-47365774-fe6c-4f10-86fa-ee45194cf368'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.183739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 969/1927] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:13 2022] DELETE /identity/v3/users/94428745298c4554af22d038dbca1c9e => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:14.185299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.188465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:14.261615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dac9e-2f2b-4c54-8ae1-6d5b0d8170e4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.270556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.271932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.272141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.272532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.273772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.276771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.296309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 970/1928] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:14.299404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.309372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699fbb05d96f4d1d8523432349c584bf', 'request_id': 'req-47365774-fe6c-4f10-86fa-ee45194cf368'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.310192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64468d3-e892-4db6-b93c-d43e4b1ca2b3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.317766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1a49b-81a8-4732-98e4-8f9c26669e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.319314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1a49b-81a8-4732-98e4-8f9c26669e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.319774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1a49b-81a8-4732-98e4-8f9c26669e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.320219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1a49b-81a8-4732-98e4-8f9c26669e36 admin admin] PATH_INFO: `/v3/projects/d64cee7a9e3f48469888895651dd76b4/users/c1849a4b1c324106a9efce7ad1dc0d15/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.323624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1a49b-81a8-4732-98e4-8f9c26669e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64cee7a9e3f48469888895651dd76b4, user_id=c1849a4b1c324106a9efce7ad1dc0d15, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.328102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1a49b-81a8-4732-98e4-8f9c26669e36 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.338219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] The token cache is being invalidate because project 699fbb05d96f4d1d8523432349c584bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:14.338522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.344161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-47365774-fe6c-4f10-86fa-ee45194cf368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.345950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 959/1929] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:13 2022] DELETE /identity/v3/projects/699fbb05d96f4d1d8523432349c584bf => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:14.350882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c410-46ba-4bb8-afbb-3174c25707ab None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.351184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c410-46ba-4bb8-afbb-3174c25707ab None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.351466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675c410-46ba-4bb8-afbb-3174c25707ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.375366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 971/1930] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:14 2022] PUT /identity/v3/projects/d64cee7a9e3f48469888895651dd76b4/users/c1849a4b1c324106a9efce7ad1dc0d15/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:14.376259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.376583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.376873 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.386668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2675c410-46ba-4bb8-afbb-3174c25707ab None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:14.389548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2675c410-46ba-4bb8-afbb-3174c25707ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:14.412155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] MFA Rules not processed for user `c1849a4b1c324106a9efce7ad1dc0d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:14.415201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:14.522004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 960/1931] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:14.534169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.534533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.535016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.558871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:14.563862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:14.565326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:14.577113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 961/1932] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:14.583745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.598490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 972/1933] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:14.603360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c3515-2c5e-4f72-a9a2-1f5ba7456e1a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.615621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.617149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.617467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.617941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.618563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.619777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:14.620136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:14.620864 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.625173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.652553 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 962/1934] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:50:14.656982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.674739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7124a8d-fb24-4448-b9fe-aa828541fa3f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.683124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.684648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.685238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.685622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.687418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.690930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.715017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3f17f-73fa-4003-9439-a78cfba313fb None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.720089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.720804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.721088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.721364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.731705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.733170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.743848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 973/1935] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:14.785016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.786444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:14.803317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658863c2-c876-42fc-920b-36969cc54761 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.808223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.809424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.809753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.810809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] PATH_INFO: `/v3/projects/9d81aa7990454c5498824d4bfeac2f46` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.816679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d81aa7990454c5498824d4bfeac2f46)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.817140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 963/1936] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:14 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:14.817367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.819010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:14.882782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6767c6-83f0-4432-8fa2-ada50f9fc03b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:14.887005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:14.887682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:14.887966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:14.894009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:14.894009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:14.894009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:14.966475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d81aa7990454c5498824d4bfeac2f46', 'request_id': 'req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:14.986899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 964/1937] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:14 2022] POST /identity/v3/users => generated 281 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:14.986899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.038558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2b86d-9586-4ea4-9ae2-9e6e7f76a41c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.056902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d81aa7990454c5498824d4bfeac2f46', 'request_id': 'req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.063688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d81aa7990454c5498824d4bfeac2f46', 'request_id': 'req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.074214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] The token cache is being invalidate because project 9d81aa7990454c5498824d4bfeac2f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:15.074502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.075458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.077054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 974/1938] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:14 2022] DELETE /identity/v3/projects/9d81aa7990454c5498824d4bfeac2f46 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:15.083011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.084639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.094838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.095677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.096437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.096757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.100278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.156213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83699d1-11ac-4f1f-ae38-f30d5b2eb48f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.161150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.161663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.161829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.162115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.162563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.163122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a042b-2f65-4438-9b84-a4e17446a2a5 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.163296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.164989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.173056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 965/1939] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:15.174427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed360811-7969-40e7-aea8-2286d29e144c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.174721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed360811-7969-40e7-aea8-2286d29e144c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.174998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed360811-7969-40e7-aea8-2286d29e144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.180670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 975/1940] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:15.184284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.195629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5b8ee-fd90-45c6-ab75-c4019f1f540d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.206516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ed360811-7969-40e7-aea8-2286d29e144c None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:15.207762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ed360811-7969-40e7-aea8-2286d29e144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.209943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.210879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.211158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.211418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.212148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.214100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.228358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 976/1941] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:15.230751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.238291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 966/1942] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:15.247077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.247569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.247838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.249000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceea9e-68df-4045-8031-d7f559fa9fd5 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.261170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a615a-e1d1-4020-8ec3-9fe598001d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.262113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a615a-e1d1-4020-8ec3-9fe598001d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.262403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a615a-e1d1-4020-8ec3-9fe598001d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.262682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a615a-e1d1-4020-8ec3-9fe598001d22 admin admin] PATH_INFO: `/v3/projects/8fafc47daeba4ed59b46f72b5af02634/users/ea628ba6dba745da936bc446378e8370/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.267112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:15.268430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.277842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 967/1943] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:15.279177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a615a-e1d1-4020-8ec3-9fe598001d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fafc47daeba4ed59b46f72b5af02634, user_id=ea628ba6dba745da936bc446378e8370, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.282888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.283025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a615a-e1d1-4020-8ec3-9fe598001d22 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.292100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5c091-9b3e-4b03-b859-a81db56af9c4 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.297084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.297746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.298030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.298308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.299045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:15.299354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.301023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.312934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 968/1944] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:50:15.317221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.327733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df86d4-f487-43a8-8cdb-85bae63ac634 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.337939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.338698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.338989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.339269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.340163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.342312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.381028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 977/1945] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:15 2022] PUT /identity/v3/projects/8fafc47daeba4ed59b46f72b5af02634/users/ea628ba6dba745da936bc446378e8370/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:15.391219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.391553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.391835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.425661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] MFA Rules not processed for user `ea628ba6dba745da936bc446378e8370`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:15.427168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.479287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 969/1946] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:15 2022] POST /identity/v3/projects => generated 351 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:15.482952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.504028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 978/1947] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:15.540904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.542640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.563498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376057ea-5344-4774-9610-ec0845f7ef76 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.571761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.572454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.572952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.573235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.573935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.576394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.610656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65055c88-9be4-4746-97e4-9bfc1977e6d6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.657776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.658425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.658705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.658980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.664471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 970/1948] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:15 2022] POST /identity/v3/users => generated 276 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:15.668835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.679511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7beaf-2208-4001-9bef-faab44356d3b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.684184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.684862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.685146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.689149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.689149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.689149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.697738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 971/1949] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:50:15.701609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.711279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42363545-90f9-4855-b745-c97133d105f0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.716377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.717087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.717391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.717694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.719453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.720741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.724782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.726222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.735144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 972/1950] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:50:15.736082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.736516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 979/1951] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:15.747874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f29ef-1523-473c-819b-85eac44f3c9f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.750314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.753126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.754023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.754304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.754582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] PATH_INFO: `/v3/projects/7f96dde281464da29b8544be1985d236/users/b390d22f088141a192d240f77d4b1503/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.764217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d740a-c1cd-46cc-a3e2-ced3542c53e9 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.769330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.770327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.770618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.771005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] PATH_INFO: `/v3/users/ea628ba6dba745da936bc446378e8370` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.772120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea628ba6dba745da936bc446378e8370)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.772665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f96dde281464da29b8544be1985d236, user_id=b390d22f088141a192d240f77d4b1503, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.774269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.775612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:15.911881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 973/1952] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:15 2022] PUT /identity/v3/projects/7f96dde281464da29b8544be1985d236/users/b390d22f088141a192d240f77d4b1503/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:15.920822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:15.920822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:15.984831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31812b4f-a706-4a29-9793-4e8209649d6b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:15.989515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:15.990327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:15.990606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:15.990877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] PATH_INFO: `/v3/users/e4e868ba0ae7450db85857f0ac84f831` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:15.991575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea628ba6dba745da936bc446378e8370'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.992104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e868ba0ae7450db85857f0ac84f831)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:15.992227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea628ba6dba745da936bc446378e8370'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.992884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea628ba6dba745da936bc446378e8370'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:15.994064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.004986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea628ba6dba745da936bc446378e8370'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.011846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f336c64b-6d5a-466c-baa8-e33cdae356ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea628ba6dba745da936bc446378e8370'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.080487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 980/1953] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:15 2022] DELETE /identity/v3/users/ea628ba6dba745da936bc446378e8370 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:16.082077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0fe1-ce0f-4710-b9f9-326b2fdcc69d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.082438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0fe1-ce0f-4710-b9f9-326b2fdcc69d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.082720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0fe1-ce0f-4710-b9f9-326b2fdcc69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.110225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-532e0fe1-ce0f-4710-b9f9-326b2fdcc69d None None] MFA Rules not processed for user `b390d22f088141a192d240f77d4b1503`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:16.111509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-532e0fe1-ce0f-4710-b9f9-326b2fdcc69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.143191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:16.174312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 981/1954] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3218 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:16.175607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.175899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.176180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.192207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e868ba0ae7450db85857f0ac84f831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.193077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e868ba0ae7450db85857f0ac84f831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.194103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:16.195303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.201601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e868ba0ae7450db85857f0ac84f831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.202474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e868ba0ae7450db85857f0ac84f831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.254180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 982/1955] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:16.254919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.256155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.267964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205af15-be11-4e7f-96b7-fbdbeb849f7f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.272750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af703b6d-a161-4bd5-a334-4534162eb86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.273478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af703b6d-a161-4bd5-a334-4534162eb86b admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.273738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af703b6d-a161-4bd5-a334-4534162eb86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.274065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af703b6d-a161-4bd5-a334-4534162eb86b admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.274757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af703b6d-a161-4bd5-a334-4534162eb86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.276689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af703b6d-a161-4bd5-a334-4534162eb86b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.279812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e868ba0ae7450db85857f0ac84f831'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:16.303289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 974/1956] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:15 2022] DELETE /identity/v3/users/e4e868ba0ae7450db85857f0ac84f831 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:16.304292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.305783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.366252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757bd7e-3f26-4ec9-b2b0-5cbc8f50f939 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.369921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.370716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.371430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.371430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.381648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 983/1957] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:16 2022] POST /identity/v3/projects => generated 357 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:16.383371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.383985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.385629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:16.386126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.408244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:16.409844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.448950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.450843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.462468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 975/1958] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:16.463287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.474324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457cb07-cd11-4fe6-8aab-4886dd421a37 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.493206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 984/1959] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:16.503006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.505012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.550397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.551979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.551979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.551979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.552284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.554262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.588629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a36140-9bf4-4e13-a047-536381003e25 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.592457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.593816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.594108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.594388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.604544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.606080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.616363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 985/1960] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:16.663947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 976/1961] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:16 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:16.668352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.678939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ad5bc-b824-417c-b932-18cb914f214c service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.684723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.685108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.685853 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.686587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.688286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.689917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:16.693876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.713961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 986/1962] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:16.714396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.736083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51381fb-332b-4e48-b03e-4f2eed70f3f3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.740564 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.741535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.741821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.742111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] PATH_INFO: `/v3/projects/d2e5fc97cd9e49e2b59856e06a099b83/users/e1e7a82737b849ad982eb2a82e682e4a/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.754966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:16.762137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e5fc97cd9e49e2b59856e06a099b83, user_id=e1e7a82737b849ad982eb2a82e682e4a, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.768729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.868619 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 977/1963] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:16 2022] PUT /identity/v3/projects/d2e5fc97cd9e49e2b59856e06a099b83/users/e1e7a82737b849ad982eb2a82e682e4a/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:16.877815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.888416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb874866-1489-40e5-af1a-e3fe5896a2e3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:16.947947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:16.948718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:16.949001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:16.949280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:16.950022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:16.951842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:16.962968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 987/1964] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:16.976312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.976673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:16.980536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:16.990388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26196eeb-e6bd-41ab-aa04-a5656614a54e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.000213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:17.000453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a3a80-07c8-4fcb-a6f5-087aeba8ebb6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.003524 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2278edc-b2c9-4f69-a9ac-b9344c807e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.005227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278edc-b2c9-4f69-a9ac-b9344c807e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.005519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278edc-b2c9-4f69-a9ac-b9344c807e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.005801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2278edc-b2c9-4f69-a9ac-b9344c807e29 admin admin] PATH_INFO: `/v3/projects/d2e5fc97cd9e49e2b59856e06a099b83/users/e1e7a82737b849ad982eb2a82e682e4a/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.006179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.007582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.008072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.008444 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] PATH_INFO: `/v3/projects/8fafc47daeba4ed59b46f72b5af02634` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.010206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fafc47daeba4ed59b46f72b5af02634)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.011375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2278edc-b2c9-4f69-a9ac-b9344c807e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e5fc97cd9e49e2b59856e06a099b83, user_id=e1e7a82737b849ad982eb2a82e682e4a, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.012432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.014107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2278edc-b2c9-4f69-a9ac-b9344c807e29 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.058247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 978/1965] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:16 2022] PUT /identity/v3/projects/d2e5fc97cd9e49e2b59856e06a099b83/users/e1e7a82737b849ad982eb2a82e682e4a/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:17.060399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.060785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.061072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.101358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] MFA Rules not processed for user `e1e7a82737b849ad982eb2a82e682e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:17.102626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.160345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fafc47daeba4ed59b46f72b5af02634', 'request_id': 'req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.176961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 979/1966] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:17.183695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.185275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.254047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca2621-2976-46bf-ac7e-054cb4f95af3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.308689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.309558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.309844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.310121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] PATH_INFO: `/v3/projects/edf48691aca94a17907346887ef5fa85` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.311301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf48691aca94a17907346887ef5fa85)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.314224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.314351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fafc47daeba4ed59b46f72b5af02634', 'request_id': 'req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.321835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fafc47daeba4ed59b46f72b5af02634', 'request_id': 'req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.334162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] The token cache is being invalidate because project 8fafc47daeba4ed59b46f72b5af02634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:17.334558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.335481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.337245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 988/1967] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:16 2022] DELETE /identity/v3/projects/8fafc47daeba4ed59b46f72b5af02634 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:17.344470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.345914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.408398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a96ba3-1904-4c2e-bf6b-a3329271b1b4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.417073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf48691aca94a17907346887ef5fa85', 'request_id': 'req-ecbd53d2-da91-48e1-a89c-9e616496a97b'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.422391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf48691aca94a17907346887ef5fa85', 'request_id': 'req-ecbd53d2-da91-48e1-a89c-9e616496a97b'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.465198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.465860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.466201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.466566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.469611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:17.470586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.496916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf48691aca94a17907346887ef5fa85', 'request_id': 'req-ecbd53d2-da91-48e1-a89c-9e616496a97b'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.509682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] The token cache is being invalidate because project edf48691aca94a17907346887ef5fa85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:17.509970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.510825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecbd53d2-da91-48e1-a89c-9e616496a97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.512808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 980/1968] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:17 2022] DELETE /identity/v3/projects/edf48691aca94a17907346887ef5fa85 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:17.514082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32928b0-6439-4cc5-bafe-b8790008866f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.514366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32928b0-6439-4cc5-bafe-b8790008866f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.515213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32928b0-6439-4cc5-bafe-b8790008866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.537534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c32928b0-6439-4cc5-bafe-b8790008866f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:17.541729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c32928b0-6439-4cc5-bafe-b8790008866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.546189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.547973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.558550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 989/1969] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:17.559587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.561096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.606451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 981/1970] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:17.615332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.615815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.616309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.637824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7c14a-e679-4baf-9baf-8dc79dd8bf66 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.638339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:17.639731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.643354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.644241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.644542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.644891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] PATH_INFO: `/v3/users/c1849a4b1c324106a9efce7ad1dc0d15` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.646018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1849a4b1c324106a9efce7ad1dc0d15)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.647962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.648450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 982/1971] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:17.653383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.665798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963880b-03af-4af1-9aa9-916f6d4a455d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.671647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:17.672340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.673242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.673522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.673795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.675035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:17.675561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.678441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.693730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 983/1972] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:50:17.697952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.710860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c91cf3-4400-4f91-9d9c-2ff3bcb6155b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.716758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.717405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.717686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.718504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.719244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.721211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:17.853399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 984/1973] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:17 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:17.857208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.860013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.886811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1849a4b1c324106a9efce7ad1dc0d15'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.887489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1849a4b1c324106a9efce7ad1dc0d15'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.888106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1849a4b1c324106a9efce7ad1dc0d15'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.909911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1849a4b1c324106a9efce7ad1dc0d15'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.917816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1849a4b1c324106a9efce7ad1dc0d15'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:17.923693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffddae-6f62-4c3f-a0f9-fcde8bec2a6e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:17.927368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:17.928020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:17.928302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:17.928610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:17.930660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.977422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 990/1974] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:17 2022] DELETE /identity/v3/users/c1849a4b1c324106a9efce7ad1dc0d15 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:17.978248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:17.979591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:17.994895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:17.996544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.006703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 985/1975] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3218 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:18.053712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:18.056934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a93cff-cf91-4909-a75c-d960ac2ddb7b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:18.062615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:18.063312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.063630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.063914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.064680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:18.066670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.155037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 991/1976] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:17 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:18.160303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:18.240264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206b8a4-4633-4634-ad35-4663cab4e5fe service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:18.252171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:18.253079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.253398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.253869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.254612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:18.256754 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.268668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 986/1977] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:18.272549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:18.284424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dbb7d-8313-4d5d-ae4e-fc1c7116a076 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:18.289527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8176e-d7cc-455b-80d0-59871d8095d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:18.290327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8176e-d7cc-455b-80d0-59871d8095d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.290620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8176e-d7cc-455b-80d0-59871d8095d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.290884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8176e-d7cc-455b-80d0-59871d8095d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.291618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8176e-d7cc-455b-80d0-59871d8095d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:18.293303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8176e-d7cc-455b-80d0-59871d8095d4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.307688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 992/1978] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:18.309243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:18.320103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f9a8-4141-4d86-a81f-3ec37172a29e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:18.324979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:18.325811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.326098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.326771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] PATH_INFO: `/v3/projects/31a8d370fefa40278d50e044e3d7d233/users/42f7a8af3b814cb592baa5a4ef5909c3/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.343867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a8d370fefa40278d50e044e3d7d233, user_id=42f7a8af3b814cb592baa5a4ef5909c3, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:18.346963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.418131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 987/1979] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:18 2022] PUT /identity/v3/projects/31a8d370fefa40278d50e044e3d7d233/users/42f7a8af3b814cb592baa5a4ef5909c3/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:18.430526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.430821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.431110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.466255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] MFA Rules not processed for user `42f7a8af3b814cb592baa5a4ef5909c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:18.472155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:18.537724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 993/1980] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:18.550982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:18.553365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:18.556596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:18.560751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:18.561647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:18.621107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d9fa6-6d3f-4cfe-b29b-a04d25f5341e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:18.625293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92592c0-b3ef-491f-8bc8-95db8a02ccc1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:18.625947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92592c0-b3ef-491f-8bc8-95db8a02ccc1 service glance] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.626228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92592c0-b3ef-491f-8bc8-95db8a02ccc1 service glance] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.626504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92592c0-b3ef-491f-8bc8-95db8a02ccc1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.636882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92592c0-b3ef-491f-8bc8-95db8a02ccc1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:18.638331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92592c0-b3ef-491f-8bc8-95db8a02ccc1 service glance] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.639972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12826a-a0b5-494b-b7f6-9bed053e2c0c None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:18.650856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:18.652309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.652859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.653316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] PATH_INFO: `/v3/projects/d64cee7a9e3f48469888895651dd76b4` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.656230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64cee7a9e3f48469888895651dd76b4)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:18.657634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 988/1981] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:18.665502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:18.770612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64cee7a9e3f48469888895651dd76b4', 'request_id': 'req-dc49c25d-2a88-4e19-a856-2407b3a550ec'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:18.840037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64cee7a9e3f48469888895651dd76b4', 'request_id': 'req-dc49c25d-2a88-4e19-a856-2407b3a550ec'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:18.847947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64cee7a9e3f48469888895651dd76b4', 'request_id': 'req-dc49c25d-2a88-4e19-a856-2407b3a550ec'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:18.852404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:18.859093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] The token cache is being invalidate because project d64cee7a9e3f48469888895651dd76b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:18.859412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:18.860144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-dc49c25d-2a88-4e19-a856-2407b3a550ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:18.868795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 994/1982] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:18 2022] DELETE /identity/v3/projects/d64cee7a9e3f48469888895651dd76b4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:18.868892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.868892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.868892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.890779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:18.891988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:18.963981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 989/1983] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:18.968481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:18.969440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:18.969928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:18.989979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:18.991222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:18.999932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 995/1984] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:19.003855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.006201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:19.013663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562837d3-9082-4bb9-9816-79a762e3c6d2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.018917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.019597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.019880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.020320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.021093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:19.021410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.023032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.035200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 990/1985] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:50:19.039321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.049116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db330fdc-8690-4165-913c-661fa3406f37 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.053843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.054505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.054792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.055080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.055856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.057754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.160322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:19.197691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 996/1986] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:19 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:19.201669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.212103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f230ae-f647-41bb-8f51-0e42240115ce admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.270544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.271220 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.271505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.271786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.272586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.274514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.390414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 991/1987] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:19 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:19.394763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.407885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fecc8-98d1-4808-8257-1f418d099aad admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.412601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.413287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.413583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.413872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.414610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.416333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.427605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 997/1988] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:19.431558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.443033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093fdac-eacd-47ff-a764-14abd8ce32aa admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.447799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe433cd-b6dd-445e-9c8a-0b073f5a2328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.448477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe433cd-b6dd-445e-9c8a-0b073f5a2328 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.448903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe433cd-b6dd-445e-9c8a-0b073f5a2328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.449108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe433cd-b6dd-445e-9c8a-0b073f5a2328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.450040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe433cd-b6dd-445e-9c8a-0b073f5a2328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.451448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe433cd-b6dd-445e-9c8a-0b073f5a2328 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.466133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 992/1989] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:50:19.466362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:19.467623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.478910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f8b4c-84b5-429d-b721-a15d44ba6f83 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.494599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.495517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.495826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.496164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] PATH_INFO: `/v3/projects/7cec1543909e407db0cf764480d31a32/users/10594b07de324ee7954da4ef9ce401c5/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.513574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec1543909e407db0cf764480d31a32, user_id=10594b07de324ee7954da4ef9ce401c5, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.516972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.607634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 998/1990] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:19 2022] PUT /identity/v3/projects/7cec1543909e407db0cf764480d31a32/users/10594b07de324ee7954da4ef9ce401c5/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:19.628936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.629240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.629533 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.675552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] MFA Rules not processed for user `10594b07de324ee7954da4ef9ce401c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:19.677075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:19.737619 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:19.762974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 993/1991] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:19.774415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:19.776106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:19.846217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad54cb-91b0-46bf-a38f-dd5d527d1706 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:19.855027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92762fbe-b337-474b-a1fa-ef35f14827f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:19.855027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92762fbe-b337-474b-a1fa-ef35f14827f1 service glance] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:19.855027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92762fbe-b337-474b-a1fa-ef35f14827f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:19.855027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92762fbe-b337-474b-a1fa-ef35f14827f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:19.863942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92762fbe-b337-474b-a1fa-ef35f14827f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:19.865691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92762fbe-b337-474b-a1fa-ef35f14827f1 service glance] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:19.886298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 999/1992] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:20.288136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:20.789352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:20.887492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:21.400404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:21.685829 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:22.353996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:22.519921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:22.816968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:32.891456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:32.913624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:46.305736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:46.307378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:46.499678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd200de8-bd9a-4352-8cca-e2482ad4d136 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:46.506633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:46.507711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:46.508056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:46.508342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] PATH_INFO: `/v3/users/588a9c687722442bb3cb3868c8866158` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:46.509773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] RBAC: Authorizing `identity:delete_user(user_id=588a9c687722442bb3cb3868c8866158)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:46.515038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:46.703391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588a9c687722442bb3cb3868c8866158'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:46.704094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588a9c687722442bb3cb3868c8866158'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:46.739719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588a9c687722442bb3cb3868c8866158'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:46.741606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588a9c687722442bb3cb3868c8866158'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:46.857237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588a9c687722442bb3cb3868c8866158'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:46.890167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 994/1993] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:46 2022] DELETE /identity/v3/users/588a9c687722442bb3cb3868c8866158 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:46.892960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:46.893454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:46.893534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:46.933476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:46.934866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:47.021507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1000/1994] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:47.028075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:47.029613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:47.092540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:47.160373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceac15-9e3d-4ac9-beaf-5515193142af admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:47.164176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89716d43-f68a-425e-859b-6be03e682494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:47.164868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89716d43-f68a-425e-859b-6be03e682494 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:47.165337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89716d43-f68a-425e-859b-6be03e682494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:47.165621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89716d43-f68a-425e-859b-6be03e682494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:47.175188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89716d43-f68a-425e-859b-6be03e682494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:47.177414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89716d43-f68a-425e-859b-6be03e682494 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:47.189983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 995/1995] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:47.727338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:47.728215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:47.743909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d5a6-d123-4818-9ab9-df7aac3fdae2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:47.757250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:47.757250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:47.757250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:47.757250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] PATH_INFO: `/v3/projects/4401a3ac9ae24ff291ed2f58f8b8d31d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:47.762115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4401a3ac9ae24ff291ed2f58f8b8d31d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:47.764667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:47.901941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4401a3ac9ae24ff291ed2f58f8b8d31d', 'request_id': 'req-6c9c5171-cfea-4e31-a91b-e6a6465682c2'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:47.999408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4401a3ac9ae24ff291ed2f58f8b8d31d', 'request_id': 'req-6c9c5171-cfea-4e31-a91b-e6a6465682c2'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:48.006481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4401a3ac9ae24ff291ed2f58f8b8d31d', 'request_id': 'req-6c9c5171-cfea-4e31-a91b-e6a6465682c2'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:48.017340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] The token cache is being invalidate because project 4401a3ac9ae24ff291ed2f58f8b8d31d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:48.017635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:48.018293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6c9c5171-cfea-4e31-a91b-e6a6465682c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:48.020463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1001/1996] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:47 2022] DELETE /identity/v3/projects/4401a3ac9ae24ff291ed2f58f8b8d31d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:48.025353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.025854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.025946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.054709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:48.056009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:48.122959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 996/1997] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:48.130903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.131203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.131493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.153326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:48.154537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:48.164929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1002/1998] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:48.169937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.182017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147c2b1-984f-4f70-bb2d-ae95d9cd1261 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.188463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.190184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.190184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.190184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.190549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:48.190908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.193519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.211496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 997/1999] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:50:48.216466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.226992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39423-e32d-4015-b43c-d12b4e8ee4ea None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.232296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.232938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.233220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.233496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.234279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.236479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.350185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1003/2000] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:48 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:48.351367 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.364485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db96cc-6f40-4172-8e40-7927ea3097ba admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.450175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.450872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.451402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.451682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.452529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.454935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.548701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 998/2001] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:48 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:48.553407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.563932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b0c40-1796-43c4-b5ce-32886542d709 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.569991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.570663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.570948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.571231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.571958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.573734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.588454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1004/2002] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:48.589759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.601477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e751a2d-7667-4585-8683-dee68de70916 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.606329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56660e-0e53-48ea-9244-5c563f522587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.607006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56660e-0e53-48ea-9244-5c563f522587 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.607289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56660e-0e53-48ea-9244-5c563f522587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.607564 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56660e-0e53-48ea-9244-5c563f522587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.608288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56660e-0e53-48ea-9244-5c563f522587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.609943 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56660e-0e53-48ea-9244-5c563f522587 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.621506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 999/2003] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:48.625850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.639554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44677038-9a3b-4af7-9ca9-3acb0b731814 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.644125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.645746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.646071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.646349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] PATH_INFO: `/v3/projects/63afcf01a7bc4cb6a4d88d96977f7311/users/76fb4368530e4954b3e02e98da278589/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.660988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63afcf01a7bc4cb6a4d88d96977f7311, user_id=76fb4368530e4954b3e02e98da278589, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.663653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.765675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1005/2004] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:48 2022] PUT /identity/v3/projects/63afcf01a7bc4cb6a4d88d96977f7311/users/76fb4368530e4954b3e02e98da278589/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:48.771212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.771564 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.771888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.781148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:48.794759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:48.795981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:48.870248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1000/2005] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:48.883740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:48.885752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:48.900781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:48.945602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46598aa-a878-4497-a098-4a53dbdc3033 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:48.950598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a4ebe-127e-4ef1-ba87-de1b996d67f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:48.951214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a4ebe-127e-4ef1-ba87-de1b996d67f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:48.951503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a4ebe-127e-4ef1-ba87-de1b996d67f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:48.951788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a4ebe-127e-4ef1-ba87-de1b996d67f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:48.962615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a4ebe-127e-4ef1-ba87-de1b996d67f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:48.964636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a4ebe-127e-4ef1-ba87-de1b996d67f1 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:48.984654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1006/2006] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:49.932334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:49.934522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:49.958211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59631156-2ca0-49df-90a7-70f67925c926 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:49.964099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:49.965034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:49.965319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:49.965597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] PATH_INFO: `/v3/users/b390d22f088141a192d240f77d4b1503` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:49.966817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b390d22f088141a192d240f77d4b1503)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:49.968813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:50.242381 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b390d22f088141a192d240f77d4b1503'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:50.243074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b390d22f088141a192d240f77d4b1503'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:50.255781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b390d22f088141a192d240f77d4b1503'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:50.256962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b390d22f088141a192d240f77d4b1503'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:50.343413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:50.343995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:50.343995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:50.353615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-78c97c89-437c-4a34-858a-a57c7e89b53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b390d22f088141a192d240f77d4b1503'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:50.384583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] MFA Rules not processed for user `76fb4368530e4954b3e02e98da278589`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:50.386402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:50.388019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1001/2007] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:49 2022] DELETE /identity/v3/users/b390d22f088141a192d240f77d4b1503 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:50.392962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:50.393430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:50.393896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:50.416950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:50.418217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:50.483351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1007/2008] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:50:50.497105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:50.500781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:50.517978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:50.527262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1002/2009] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:50.538160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:50.539525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:50.591423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f91e2-7fa6-4d24-bae8-2ea7ea85ea61 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:50.602342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:50.603527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:50.603976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:50.605112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:50.638206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:50.648702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:50.648842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff9987-fe0d-4007-a059-a652ab727cbc None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:50.648842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:50.648842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:50.648842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:50.649148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:50.660192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:50.661249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1008/2010] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:50.663183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:50.674002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1003/2011] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:50:51.373608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:51.375143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:51.408097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e68c88-b215-4c3e-b9b3-091996866702 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:51.420694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:51.421784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:51.422222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:51.422722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] PATH_INFO: `/v3/projects/7f96dde281464da29b8544be1985d236` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:51.424018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f96dde281464da29b8544be1985d236)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:51.428772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:51.543408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:51.553804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f96dde281464da29b8544be1985d236', 'request_id': 'req-83f46374-f48c-4868-b158-e268233aea68'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.605355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f96dde281464da29b8544be1985d236', 'request_id': 'req-83f46374-f48c-4868-b158-e268233aea68'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.612348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f96dde281464da29b8544be1985d236', 'request_id': 'req-83f46374-f48c-4868-b158-e268233aea68'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.625961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] The token cache is being invalidate because project 7f96dde281464da29b8544be1985d236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:51.626286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.626931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.628568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1009/2012] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:51 2022] DELETE /identity/v3/projects/7f96dde281464da29b8544be1985d236 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:51.633579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:51.644106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59016-444a-401d-9acb-047fe4795750 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:51.721351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:51.722242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:51.722524 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:51.722830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] PATH_INFO: `/v3/users/e1e7a82737b849ad982eb2a82e682e4a` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:51.724116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e7a82737b849ad982eb2a82e682e4a)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:51.726060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:51.891816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e7a82737b849ad982eb2a82e682e4a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.892768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e7a82737b849ad982eb2a82e682e4a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.902603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e7a82737b849ad982eb2a82e682e4a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.903511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e7a82737b849ad982eb2a82e682e4a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.959745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-74478946-9729-48d6-918d-a980f46a0efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e7a82737b849ad982eb2a82e682e4a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:51.985622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1004/2013] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:51 2022] DELETE /identity/v3/users/e1e7a82737b849ad982eb2a82e682e4a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:52.344747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:52.346473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:52.371014 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46374-f48c-4868-b158-e268233aea68 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:52.466751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:52.467638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:52.467924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:52.468204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] PATH_INFO: `/v3/projects/d2e5fc97cd9e49e2b59856e06a099b83` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:52.474311 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e5fc97cd9e49e2b59856e06a099b83)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:52.476446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:52.479444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e7260-9574-423a-8bbc-e1c7ce717eed None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:52.479607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e7260-9574-423a-8bbc-e1c7ce717eed None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:52.479775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e7260-9574-423a-8bbc-e1c7ce717eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:52.508855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c58e7260-9574-423a-8bbc-e1c7ce717eed None None] MFA Rules not processed for user `76fb4368530e4954b3e02e98da278589`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:52.510139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c58e7260-9574-423a-8bbc-e1c7ce717eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:52.531314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:52.587741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1005/2014] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:52.619063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e5fc97cd9e49e2b59856e06a099b83', 'request_id': 'req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:52.686989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e5fc97cd9e49e2b59856e06a099b83', 'request_id': 'req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:52.701616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e5fc97cd9e49e2b59856e06a099b83', 'request_id': 'req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:52.728885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] The token cache is being invalidate because project d2e5fc97cd9e49e2b59856e06a099b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:52.729336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:52.730325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f5f18822-bdc6-4a4f-bbc2-c8c2e7cb872a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:52.732485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1010/2015] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:52 2022] DELETE /identity/v3/projects/d2e5fc97cd9e49e2b59856e06a099b83 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:52.738290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:52.738846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:52.739163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:52.779145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:52.781104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:52.848940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1006/2016] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:52.856958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:52.857467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:52.857571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:52.888243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:52.889564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:52.898402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1011/2017] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:52.902854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:52.913169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646f8f3-688e-483e-a973-50559b4c8a51 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:52.918787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:52.919050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:52.919144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:52.919422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:52.920601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:52.920601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:52.924281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:52.935115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1007/2018] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:50:52.940787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:52.953613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b09dec-facd-4efc-9bec-36a2252ca5a9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:52.962236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:52.963324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:52.963784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:52.964299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:52.965519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:52.968890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:53.060989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1012/2019] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:52 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:53.063003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:53.074936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8762f4b-6f1f-4680-8212-1949b6efe68a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:53.148604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:53.148604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:53.148604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:53.148604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:53.149099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:53.149099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:53.434392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1008/2020] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:53 2022] POST /identity/v3/users => generated 283 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:53.439111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:53.460348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e0496-9613-46eb-90ec-eb2028d9ace0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:53.469534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:53.470682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:53.471956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:53.473905 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:53.475324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:53.478515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:53.496800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1013/2021] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:53.501012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:53.511504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927c0a7-9529-4b9a-b2e9-e6bf8902b086 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:53.516530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be980f-c23a-4d8b-8f2c-3c1ef57c8628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:53.517128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be980f-c23a-4d8b-8f2c-3c1ef57c8628 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:53.517411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be980f-c23a-4d8b-8f2c-3c1ef57c8628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:53.517687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be980f-c23a-4d8b-8f2c-3c1ef57c8628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:53.518407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be980f-c23a-4d8b-8f2c-3c1ef57c8628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:53.520568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be980f-c23a-4d8b-8f2c-3c1ef57c8628 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:53.546198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1009/2022] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:53.550256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:53.582143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd22b10-afd1-4730-9f35-122a243078b6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:53.614075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:53.615696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:53.615993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:53.616277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] PATH_INFO: `/v3/projects/38717d870d3247dd8f8462e7ed9ddf09/users/b46d1f62008d4ac5844ea62ede2c392c/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:53.643517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:53.649336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=38717d870d3247dd8f8462e7ed9ddf09, user_id=b46d1f62008d4ac5844ea62ede2c392c, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:53.652984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:53.789348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1014/2023] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:53 2022] PUT /identity/v3/projects/38717d870d3247dd8f8462e7ed9ddf09/users/b46d1f62008d4ac5844ea62ede2c392c/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:53.800712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:53.801831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:53.801929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:53.838013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] MFA Rules not processed for user `b46d1f62008d4ac5844ea62ede2c392c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:53.839348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:53.849985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:53.911382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1010/2024] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:53.916480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:53.983480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50bf8-2f15-4a4e-a67e-663e2ba89bac admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:53.990148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:53.990825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:53.991116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:53.991401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:53.992113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:53.994598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.067623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1015/2025] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:53 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:54.069060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.081122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ef46b-4d20-42a1-a943-df4a2fb3ba89 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.142813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.143506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.143789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.144065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.145347 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:54.149111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.252549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1011/2026] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:54 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:54.256227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.266929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79d59f-66ee-40a1-9913-8be00e254748 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.273347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.274026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.274319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.274602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.275408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:54.277068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.291870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1016/2027] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:54.294025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.305221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359ca3d-faa6-42d5-b918-86d4fdbd3759 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.309971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.310898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.311186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.311467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] PATH_INFO: `/v3/projects/0f5495ab283148cca50894f0fc871cfb/users/003c026d512a46f8bcf8f72716cccaba/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.327459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5495ab283148cca50894f0fc871cfb, user_id=003c026d512a46f8bcf8f72716cccaba, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:54.330215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.434479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1012/2028] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:54 2022] PUT /identity/v3/projects/0f5495ab283148cca50894f0fc871cfb/users/003c026d512a46f8bcf8f72716cccaba/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:54.438501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.449375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972c365-319a-4626-8ca7-7910f7913e07 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.507854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6ca09-cf35-4439-8b8a-839ec89371e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.508566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6ca09-cf35-4439-8b8a-839ec89371e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.508839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6ca09-cf35-4439-8b8a-839ec89371e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.509123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6ca09-cf35-4439-8b8a-839ec89371e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.509883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6ca09-cf35-4439-8b8a-839ec89371e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:54.511498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6ca09-cf35-4439-8b8a-839ec89371e6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.522887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1017/2029] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:54.527158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.537890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee80b35-d607-4b02-bc5e-bd7563fbfda7 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.546450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.546450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.546450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.546450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] PATH_INFO: `/v3/projects/0f5495ab283148cca50894f0fc871cfb/users/003c026d512a46f8bcf8f72716cccaba/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.546899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5495ab283148cca50894f0fc871cfb, user_id=003c026d512a46f8bcf8f72716cccaba, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:54.549570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.562361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1013/2030] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:54 2022] PUT /identity/v3/projects/0f5495ab283148cca50894f0fc871cfb/users/003c026d512a46f8bcf8f72716cccaba/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:54.573929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.574255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.574636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.603071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] MFA Rules not processed for user `003c026d512a46f8bcf8f72716cccaba`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:54.604325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:54.661764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:54.698644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1018/2031] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:54.710388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.713299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:54.781584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da61a9-e006-438a-b6d9-2992e995a30d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.787132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7358b0-338d-4c27-98c3-03ae8452b511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.788071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7358b0-338d-4c27-98c3-03ae8452b511 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.788486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7358b0-338d-4c27-98c3-03ae8452b511 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.788847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7358b0-338d-4c27-98c3-03ae8452b511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.798918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7358b0-338d-4c27-98c3-03ae8452b511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:54.800409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7358b0-338d-4c27-98c3-03ae8452b511 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:54.811481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1014/2032] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:54.980202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:54.991058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8ccc8-47e1-478a-afca-0b26d697433b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:54.995116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfaedd-014d-4531-809e-948c1f6331c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:54.997497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfaedd-014d-4531-809e-948c1f6331c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:55.001491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfaedd-014d-4531-809e-948c1f6331c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:55.001491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfaedd-014d-4531-809e-948c1f6331c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:55.064247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfaedd-014d-4531-809e-948c1f6331c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:55.065647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfaedd-014d-4531-809e-948c1f6331c4 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:55.076762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1019/2033] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:57.045360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec5ae5-259f-45fa-9182-19d2b3f68c41 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:57.045867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec5ae5-259f-45fa-9182-19d2b3f68c41 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:57.045978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec5ae5-259f-45fa-9182-19d2b3f68c41 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:57.047283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1015/2034] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:50:57.060740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:57.061270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:57.061367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:57.080192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:57.081522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:57.171510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1020/2035] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:50:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:57.183321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec5ae5-259f-45fa-9182-19d2b3f68c41 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:57.195429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b7ec5ae5-259f-45fa-9182-19d2b3f68c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:57.202536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec5ae5-259f-45fa-9182-19d2b3f68c41 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:57.207973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:57.208744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:57.209030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:57.209758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:57.215994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:57.220208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:57.222502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:57.230897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:57.232808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:57.263296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1016/2036] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:50:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:57.330722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8c99b-041a-4183-b1c2-6a28498821f5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:57.336211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:57.337144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:57.337471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:57.337753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] PATH_INFO: `/v3/users/0ebc2cec00384f28bb79d0742d3bccf1` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:57.339023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ebc2cec00384f28bb79d0742d3bccf1)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:57.341056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:57.369857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:57.566285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ebc2cec00384f28bb79d0742d3bccf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:57.567698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ebc2cec00384f28bb79d0742d3bccf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:57.568584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ebc2cec00384f28bb79d0742d3bccf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:57.584771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ebc2cec00384f28bb79d0742d3bccf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:57.592404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ba92af51-48d4-4cdd-9909-ffec34e9944b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ebc2cec00384f28bb79d0742d3bccf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:57.660128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1021/2037] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:50:57 2022] DELETE /identity/v3/users/0ebc2cec00384f28bb79d0742d3bccf1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:58.045688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.057705 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999c8cd-75cd-4a30-8bf5-f3e078eb8c3e service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.122893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.123767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.124046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.124325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] PATH_INFO: `/v3/projects/7e34d34b28b44d399fe2a1e011771f22` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.125679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e34d34b28b44d399fe2a1e011771f22)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.127598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:58.258258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e34d34b28b44d399fe2a1e011771f22', 'request_id': 'req-40609fb8-ae8d-4961-8c6b-ca126575a20c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:58.264170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e34d34b28b44d399fe2a1e011771f22', 'request_id': 'req-40609fb8-ae8d-4961-8c6b-ca126575a20c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:58.359386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e34d34b28b44d399fe2a1e011771f22', 'request_id': 'req-40609fb8-ae8d-4961-8c6b-ca126575a20c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:58.372616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] The token cache is being invalidate because project 7e34d34b28b44d399fe2a1e011771f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:50:58.372867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:58.373485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-40609fb8-ae8d-4961-8c6b-ca126575a20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:50:58.375138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1017/2038] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:50:58 2022] DELETE /identity/v3/projects/7e34d34b28b44d399fe2a1e011771f22 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:58.382469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.382968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.383054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.403272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:58.404481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:58.470188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1022/2039] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:58.482798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.483090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.483373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.503531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:58.504794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:58.512614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1018/2040] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:58.517121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.528024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33965d-646c-499d-a51e-f11d6fc3b012 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.536211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.537359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.537648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.537927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.538672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:50:58.538984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.540699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:58.554021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1023/2041] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:50:58.558401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.567807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aaa14-d2ba-4880-b32a-369ae2862419 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.572392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.573067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.573351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.573623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.574310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.576233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:58.635087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1019/2042] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:58 2022] POST /identity/v3/projects => generated 409 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:58.639321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.649953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b00fc3-bf69-4a1e-9b06-1d82bc122dd1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.712405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.713136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.713426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.713708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.714431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.716603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:58.845327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1024/2043] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:58 2022] POST /identity/v3/users => generated 305 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:58.853889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.864370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901eb1e-b899-4f97-823e-21d2a9dc68f8 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.872101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.872813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.873049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.873297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.874030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.875675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:58.888004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1020/2044] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:58.892192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.903460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9073271-d6c5-45cf-914e-36b8ba0d08bf admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.908563 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc98359-9bd4-4934-9672-e5987510ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.913169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc98359-9bd4-4934-9672-e5987510ee70 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.913534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc98359-9bd4-4934-9672-e5987510ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.913924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc98359-9bd4-4934-9672-e5987510ee70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.914742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc98359-9bd4-4934-9672-e5987510ee70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.916421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc98359-9bd4-4934-9672-e5987510ee70 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:58.926953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1025/2045] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:58.931325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:58.941378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e9b57-3408-45e8-98f3-d388f6ffe963 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:58.946751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:58.947628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:58.947916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:58.948193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] PATH_INFO: `/v3/projects/7a0f8df2c54c4a8e83307d74274ccb8e/users/b2aeac1563ec4347bde9b091d74ffd67/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:58.964002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a0f8df2c54c4a8e83307d74274ccb8e, user_id=b2aeac1563ec4347bde9b091d74ffd67, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:58.966721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.120183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1021/2046] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:58 2022] PUT /identity/v3/projects/7a0f8df2c54c4a8e83307d74274ccb8e/users/b2aeac1563ec4347bde9b091d74ffd67/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:59.132257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.132554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.132834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.139542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:59.159379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] MFA Rules not processed for user `b2aeac1563ec4347bde9b091d74ffd67`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:59.161296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:59.226850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1026/2047] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:50:59.231462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:59.254963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:59.295303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f792066-8315-4a02-bbd3-3fc7fdf524d0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:59.299873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:59.300622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.301411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.301688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.302382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:59.304329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.418207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1022/2048] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:50:59 2022] POST /identity/v3/projects => generated 409 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:50:59.423155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:59.435123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af266ea6-b79d-4ed3-a653-942ab13d9ce3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:59.500721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:59.501487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.501931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.502204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.502993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:59.506092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.576598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1027/2049] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:50:59 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:50:59.580428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:59.591855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c462-f6f9-43a7-8f4e-034574938979 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:59.597079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:59.598497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.598915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.599214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.600017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:59.601734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.612836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1023/2050] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:59.617736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:59.627926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d0620-fbf0-46cb-980d-16529a36193e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:59.633306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:59.634190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.634506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.634788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] PATH_INFO: `/v3/projects/ccbcc8416dc1452bb0966d71a75ebd85/users/01d85da9aa534c36b668fc38464a1287/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.650169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbcc8416dc1452bb0966d71a75ebd85, user_id=01d85da9aa534c36b668fc38464a1287, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:59.652810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.779055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1028/2051] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:59 2022] PUT /identity/v3/projects/ccbcc8416dc1452bb0966d71a75ebd85/users/01d85da9aa534c36b668fc38464a1287/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:50:59.783353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:59.795016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16f0e4-3204-447b-8684-7e55f34b584f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:59.852974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eece55b-0b11-4a42-9e50-4cbffda0cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:59.853666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eece55b-0b11-4a42-9e50-4cbffda0cc1b admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.853954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eece55b-0b11-4a42-9e50-4cbffda0cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.854234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eece55b-0b11-4a42-9e50-4cbffda0cc1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.854992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eece55b-0b11-4a42-9e50-4cbffda0cc1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:59.856621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eece55b-0b11-4a42-9e50-4cbffda0cc1b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.872222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1024/2052] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:50:59.873769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:59.887499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354e725-b7cb-4f2c-bb4f-36d2567506a5 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:59.892745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:59.895009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.895009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.895009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] PATH_INFO: `/v3/projects/ccbcc8416dc1452bb0966d71a75ebd85/users/01d85da9aa534c36b668fc38464a1287/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.897097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbcc8416dc1452bb0966d71a75ebd85, user_id=01d85da9aa534c36b668fc38464a1287, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:59.900093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:59.919422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1029/2053] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:50:59 2022] PUT /identity/v3/projects/ccbcc8416dc1452bb0966d71a75ebd85/users/01d85da9aa534c36b668fc38464a1287/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:50:59.930853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:59.931153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:59.931436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:59.972479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] MFA Rules not processed for user `01d85da9aa534c36b668fc38464a1287`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:59.975391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:00.074479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1025/2054] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:00.085805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:00.087596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:00.143452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:00.149462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae820d-90bd-42aa-9534-d144f8c43ac8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:00.153439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:00.154032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:00.154320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:00.154660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:00.164779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:00.168119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:00.175852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1030/2055] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:00.596278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:00.610138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6cbbf-80f3-42c6-8cb9-8b79b86d95bd None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:00.616342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0e259-9323-487a-9a80-6eb9fcad2ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:00.617093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0e259-9323-487a-9a80-6eb9fcad2ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:00.617449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0e259-9323-487a-9a80-6eb9fcad2ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:00.617796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0e259-9323-487a-9a80-6eb9fcad2ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:00.632427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:00.686583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0e259-9323-487a-9a80-6eb9fcad2ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:00.688062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0e259-9323-487a-9a80-6eb9fcad2ef9 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:00.697607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1026/2056] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:10.530608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:10.753786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=21 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:21.065318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:21.066770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:21.143470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167fe94-2206-4642-b054-162933fd8b45 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:21.148657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:21.149555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:21.149835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:21.150114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] PATH_INFO: `/v3/users/b46d1f62008d4ac5844ea62ede2c392c` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:21.151318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46d1f62008d4ac5844ea62ede2c392c)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:21.153330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:21.328607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46d1f62008d4ac5844ea62ede2c392c'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:21.329286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46d1f62008d4ac5844ea62ede2c392c'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:21.329896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46d1f62008d4ac5844ea62ede2c392c'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:21.342878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46d1f62008d4ac5844ea62ede2c392c'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:21.349943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46d1f62008d4ac5844ea62ede2c392c'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:21.410472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1031/2057] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:21 2022] DELETE /identity/v3/users/b46d1f62008d4ac5844ea62ede2c392c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:21.415176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:21.415669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:21.415765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:21.434620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:21.435910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:21.496760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1027/2058] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:21.506784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:21.509501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:21.568643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f0048-83e7-475a-842c-ece5d280c5b6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:21.573628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:21.574285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:21.574680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:21.574994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:21.584724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:21.586803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:21.596623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1032/2059] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:22.019583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:22.021252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:22.035198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ba5d7-a528-455d-b199-27eca8998305 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:22.040695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:22.041589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:22.041872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:22.042147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] PATH_INFO: `/v3/projects/38717d870d3247dd8f8462e7ed9ddf09` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:22.043357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38717d870d3247dd8f8462e7ed9ddf09)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:22.045302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:22.151239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38717d870d3247dd8f8462e7ed9ddf09', 'request_id': 'req-097a04b1-a3e4-43ad-9bda-37ae95ce7062'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.157036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38717d870d3247dd8f8462e7ed9ddf09', 'request_id': 'req-097a04b1-a3e4-43ad-9bda-37ae95ce7062'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.256365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38717d870d3247dd8f8462e7ed9ddf09', 'request_id': 'req-097a04b1-a3e4-43ad-9bda-37ae95ce7062'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.269098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] The token cache is being invalidate because project 38717d870d3247dd8f8462e7ed9ddf09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:22.269394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.270067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.271622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1028/2060] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:22 2022] DELETE /identity/v3/projects/38717d870d3247dd8f8462e7ed9ddf09 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:22.276415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:22.286310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf806abb-4d94-4596-8db8-3ade211bde41 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:22.306148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:22.345536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:22.346550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:22.346843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:22.347274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] PATH_INFO: `/v3/users/003c026d512a46f8bcf8f72716cccaba` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:22.348602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=003c026d512a46f8bcf8f72716cccaba)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:22.350565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:22.431760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:22.434869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:22.449580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a04b1-a3e4-43ad-9bda-37ae95ce7062 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:22.455312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:22.456466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:22.456841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:22.457220 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] PATH_INFO: `/v3/users/b2aeac1563ec4347bde9b091d74ffd67` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:22.458729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2aeac1563ec4347bde9b091d74ffd67)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:22.462469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:22.578871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003c026d512a46f8bcf8f72716cccaba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.579454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003c026d512a46f8bcf8f72716cccaba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.580078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003c026d512a46f8bcf8f72716cccaba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.592753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003c026d512a46f8bcf8f72716cccaba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.599958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c97df7c7-c044-40f9-8708-37c4f0fb8c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003c026d512a46f8bcf8f72716cccaba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.701134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2aeac1563ec4347bde9b091d74ffd67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.701569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2aeac1563ec4347bde9b091d74ffd67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.709625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2aeac1563ec4347bde9b091d74ffd67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.710351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2aeac1563ec4347bde9b091d74ffd67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.715580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1033/2061] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:22 2022] DELETE /identity/v3/users/003c026d512a46f8bcf8f72716cccaba => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:22.816735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2aeac1563ec4347bde9b091d74ffd67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:22.831372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:22.837042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1029/2062] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:22 2022] DELETE /identity/v3/users/b2aeac1563ec4347bde9b091d74ffd67 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:22.841778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:22.842069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:22.842347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:22.860820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:22.865917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:22.938957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1034/2063] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:22.946385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:22.947917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:23.009602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac2a4e-ba87-46a6-a515-183eac820c70 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:23.013394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f60fb8-dcad-4737-b1ab-6a62eafc5ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:23.014047 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f60fb8-dcad-4737-b1ab-6a62eafc5ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.014327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f60fb8-dcad-4737-b1ab-6a62eafc5ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.015238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f60fb8-dcad-4737-b1ab-6a62eafc5ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.024985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f60fb8-dcad-4737-b1ab-6a62eafc5ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:23.026631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f60fb8-dcad-4737-b1ab-6a62eafc5ce6 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:23.037689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1030/2064] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:51:23.183445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:23.187423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:23.198870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0c470-7189-4941-be2d-001ea824ba0a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:23.203593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:23.204472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.204789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.205065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] PATH_INFO: `/v3/projects/0f5495ab283148cca50894f0fc871cfb` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.206284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5495ab283148cca50894f0fc871cfb)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:23.208378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:23.348760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:23.358745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5495ab283148cca50894f0fc871cfb', 'request_id': 'req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.443960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5495ab283148cca50894f0fc871cfb', 'request_id': 'req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.451845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5495ab283148cca50894f0fc871cfb', 'request_id': 'req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.467158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] The token cache is being invalidate because project 0f5495ab283148cca50894f0fc871cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:23.467698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.468833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.480004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1035/2065] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:23 2022] DELETE /identity/v3/projects/0f5495ab283148cca50894f0fc871cfb => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:23.484483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40483-1b95-4508-bf7b-0fda8d9cf53f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.484984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40483-1b95-4508-bf7b-0fda8d9cf53f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.485075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40483-1b95-4508-bf7b-0fda8d9cf53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.514904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-90d40483-1b95-4508-bf7b-0fda8d9cf53f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:23.516237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-90d40483-1b95-4508-bf7b-0fda8d9cf53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:23.541174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:23.543619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:23.661444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1031/2066] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:23.666209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.666635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.667095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.700294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:23.701697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:23.703131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f54a8-ac61-473a-aa57-01c6f1eb50f9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:23.712187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:23.715716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1032/2067] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:23.715848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.715848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.715848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] PATH_INFO: `/v3/projects/7a0f8df2c54c4a8e83307d74274ccb8e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.715848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a0f8df2c54c4a8e83307d74274ccb8e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:23.717254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:23.717470 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:23.729200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c21f-4d92-4954-b256-5a28baa814f3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:23.734608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:23.735434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.735658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.735990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.736879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:23.737125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:23.738770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:23.752807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1033/2068] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:51:23.756997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:23.774899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf242-2fb4-4151-9612-8c6c60ea0d13 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:23.782571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:23.783382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:23.783764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:23.784132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:23.784936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:23.788081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:23.857976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a0f8df2c54c4a8e83307d74274ccb8e', 'request_id': 'req-5312d72f-6ba1-4781-88fb-2071165534d6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.926574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1034/2069] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:23 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:23.931422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:23.978481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a0f8df2c54c4a8e83307d74274ccb8e', 'request_id': 'req-5312d72f-6ba1-4781-88fb-2071165534d6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:23.990027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:23.993521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a0f8df2c54c4a8e83307d74274ccb8e', 'request_id': 'req-5312d72f-6ba1-4781-88fb-2071165534d6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.002686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6f63c-bbbc-40c3-9822-dcf56fb68585 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:24.007816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:24.008744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.009061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.009312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.010011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:24.011428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] The token cache is being invalidate because project 7a0f8df2c54c4a8e83307d74274ccb8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:24.011614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.012105 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:24.012435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.015307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1036/2070] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:23 2022] DELETE /identity/v3/projects/7a0f8df2c54c4a8e83307d74274ccb8e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:24.019537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:24.021476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:24.093386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:24.098413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d72f-6ba1-4781-88fb-2071165534d6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:24.103879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:24.104781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.105063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.105337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] PATH_INFO: `/v3/users/01d85da9aa534c36b668fc38464a1287` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.106531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d85da9aa534c36b668fc38464a1287)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:24.108470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:24.228452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1035/2071] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:23 2022] POST /identity/v3/users => generated 295 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:24.232322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:24.234031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:24.246814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895733a6-4054-479c-866b-fce8ea54eae2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:24.251251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:24.301857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.301857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.301857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.301857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:24.301857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:24.301857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1036/2072] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:24.304597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:24.314325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eedea6-9d36-4f11-bf2b-0f2403c7e754 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:24.318844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:24.319499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.319787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.320075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.320908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:24.322436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:24.334169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1037/2073] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:24.338900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:24.349488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877301b-1245-45aa-ba23-6ab10688488d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:24.355873 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a762d0-08d1-4f1d-a13b-b0e28554600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:24.356814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a762d0-08d1-4f1d-a13b-b0e28554600f admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.357126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a762d0-08d1-4f1d-a13b-b0e28554600f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.357405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a762d0-08d1-4f1d-a13b-b0e28554600f admin admin] PATH_INFO: `/v3/projects/6e9a5f8224b440c6b1b9e9a6b366fae7/users/47d78cb72bd74e8087c6cfca67d6f263/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.373364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a762d0-08d1-4f1d-a13b-b0e28554600f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a5f8224b440c6b1b9e9a6b366fae7, user_id=47d78cb72bd74e8087c6cfca67d6f263, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:24.377398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a762d0-08d1-4f1d-a13b-b0e28554600f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:24.539753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1038/2074] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:24 2022] PUT /identity/v3/projects/6e9a5f8224b440c6b1b9e9a6b366fae7/users/47d78cb72bd74e8087c6cfca67d6f263/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:24.545771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.546090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.546505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.546801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d85da9aa534c36b668fc38464a1287'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.547632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d85da9aa534c36b668fc38464a1287'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.548307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d85da9aa534c36b668fc38464a1287'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.561956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d85da9aa534c36b668fc38464a1287'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.571811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d85da9aa534c36b668fc38464a1287'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:24.581526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] MFA Rules not processed for user `47d78cb72bd74e8087c6cfca67d6f263`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:24.583150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:24.667216 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1039/2075] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:24.667521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1037/2076] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:24 2022] DELETE /identity/v3/users/01d85da9aa534c36b668fc38464a1287 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:24.667624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:24.669359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:24.741178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a19387-3669-4241-a7ab-60d396d01a1b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:24.771026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:24.801510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:24.802173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:24.802460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:24.802748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:24.803467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:24.805447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:24.933779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1040/2077] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:24 2022] POST /identity/v3/projects => generated 389 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:24.937627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:24.955352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a9c1-8d6f-49e3-84aa-564ab4646603 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.036156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.036868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.037165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.037453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.038165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.040384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.159576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1038/2078] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:24 2022] POST /identity/v3/users => generated 301 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:25.163694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.175097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2176b-1e4c-4ac1-8f16-b902a3ce3fb5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.180370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.181098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.181389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.181671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.182403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.184049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.195921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1041/2079] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:25.200147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.201385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.203532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.211885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c592de-0d7b-4ee4-8dd2-09a1d4bf624d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.219792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a372834-272a-46be-84f6-80c501f68606 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.222177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.223168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.223530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.223934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] PATH_INFO: `/v3/projects/2bfa456a860447289ba58defabf8003d/users/f705d5aa8523493899c670a524b7fe37/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.227958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.231914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.231914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.231914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] PATH_INFO: `/v3/projects/ccbcc8416dc1452bb0966d71a75ebd85` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.231914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccbcc8416dc1452bb0966d71a75ebd85)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.233045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.241449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfa456a860447289ba58defabf8003d, user_id=f705d5aa8523493899c670a524b7fe37, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.244667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.326019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1039/2080] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:25 2022] PUT /identity/v3/projects/2bfa456a860447289ba58defabf8003d/users/f705d5aa8523493899c670a524b7fe37/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:25.329850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.361353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccbcc8416dc1452bb0966d71a75ebd85', 'request_id': 'req-6009439e-3c33-426f-bedc-5878e5a8b328'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.367328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccbcc8416dc1452bb0966d71a75ebd85', 'request_id': 'req-6009439e-3c33-426f-bedc-5878e5a8b328'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.374720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:25.394278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b639e34-5e36-402a-af57-ee3567f12c55 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.427521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccbcc8416dc1452bb0966d71a75ebd85', 'request_id': 'req-6009439e-3c33-426f-bedc-5878e5a8b328'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.440707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] The token cache is being invalidate because project ccbcc8416dc1452bb0966d71a75ebd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:25.440997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.442516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-6009439e-3c33-426f-bedc-5878e5a8b328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.443210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1042/2081] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:25 2022] DELETE /identity/v3/projects/ccbcc8416dc1452bb0966d71a75ebd85 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:25.449468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324a6f0-4c45-41f9-a892-1aa6ae6acf76 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.449757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324a6f0-4c45-41f9-a892-1aa6ae6acf76 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.450094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324a6f0-4c45-41f9-a892-1aa6ae6acf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.450556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.451226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.451520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.451801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.452558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.454165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.464624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1040/2082] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:25.468883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.469028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7324a6f0-4c45-41f9-a892-1aa6ae6acf76 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:25.470314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7324a6f0-4c45-41f9-a892-1aa6ae6acf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.470615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.483846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1043/2083] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:25.486291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb6cf2-768c-4c47-ab98-935b74ce2673 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.490946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60546-f8af-47b6-9b91-3935eea54fcb None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.491232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60546-f8af-47b6-9b91-3935eea54fcb None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.491507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf60546-f8af-47b6-9b91-3935eea54fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.494652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.495665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.495956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.496248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] PATH_INFO: `/v3/projects/2bfa456a860447289ba58defabf8003d/users/f705d5aa8523493899c670a524b7fe37/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.498653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfa456a860447289ba58defabf8003d, user_id=f705d5aa8523493899c670a524b7fe37, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.501646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.515216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1041/2084] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:25 2022] PUT /identity/v3/projects/2bfa456a860447289ba58defabf8003d/users/f705d5aa8523493899c670a524b7fe37/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:25.517182 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.517714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.525109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bcf60546-f8af-47b6-9b91-3935eea54fcb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:25.526445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bcf60546-f8af-47b6-9b91-3935eea54fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.588039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cec623-3709-4390-b64e-71bdfe45546f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.592934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.593097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1044/2085] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:25.594007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.594228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.594437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.594558 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.594731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] PATH_INFO: `/v3/users/10594b07de324ee7954da4ef9ce401c5` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.594834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.595983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10594b07de324ee7954da4ef9ce401c5)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.598447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.626073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] MFA Rules not processed for user `f705d5aa8523493899c670a524b7fe37`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:25.627340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.638521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:25.699101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1045/2086] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:25.700460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.716050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1d284-2e2c-4428-ad01-3c66d1ae91fa None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.721314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.722002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.722280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.722552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.723364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:25.723678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.725456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.738314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1046/2087] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:51:25.739454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.742302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.800763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31792e18-7420-443e-930c-60d59d8d833c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.804466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.805353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.805632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.805901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.817069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.818587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.827931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10594b07de324ee7954da4ef9ce401c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.828462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10594b07de324ee7954da4ef9ce401c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.829127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10594b07de324ee7954da4ef9ce401c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.830496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1047/2088] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:25.831173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.843090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529269-303c-470f-8bcc-5628c3d39a1d service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.848810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5cc1e8-cfc2-43fe-8156-2f82af1080c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.849431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cc1e8-cfc2-43fe-8156-2f82af1080c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.849715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cc1e8-cfc2-43fe-8156-2f82af1080c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.849987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cc1e8-cfc2-43fe-8156-2f82af1080c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.850669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cc1e8-cfc2-43fe-8156-2f82af1080c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.860264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5cc1e8-cfc2-43fe-8156-2f82af1080c9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.860388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10594b07de324ee7954da4ef9ce401c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.868286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10594b07de324ee7954da4ef9ce401c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:25.902735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:25.976934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1042/2089] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:25 2022] DELETE /identity/v3/users/10594b07de324ee7954da4ef9ce401c5 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:25.977798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.979131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.019508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1048/2090] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:25 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:26.020886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.026240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.026240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.041177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:26.042708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.050764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5b63-6d36-4c17-99bd-cdb439de23c1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.108188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1049/2091] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:26.111447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.111447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.115333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.116427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.117422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.117854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] PATH_INFO: `/v3/users/47d78cb72bd74e8087c6cfca67d6f263` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.119320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d78cb72bd74e8087c6cfca67d6f263)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.121484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.124281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69817f98-2e34-4416-9548-b16f5c2c440b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.130915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.131605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.131889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.132936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.133948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.135923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.146334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:26.234725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1050/2092] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:26 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:26.238364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.240748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.296929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63caf-327b-4d89-b967-39668e86456d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.300816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.301469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.301760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.302037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.312097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.313589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.323862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1051/2093] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:26.324742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.335022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983648a-5425-47d8-b45d-35b7da172d6b service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.340335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.341164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.341455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.341731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.342456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.344150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.346643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d78cb72bd74e8087c6cfca67d6f263'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:26.350564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d78cb72bd74e8087c6cfca67d6f263'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:26.350564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d78cb72bd74e8087c6cfca67d6f263'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:26.355918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1052/2094] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:26.359836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.362265 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.365230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d78cb72bd74e8087c6cfca67d6f263'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:26.374791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cda032bf-e3f2-46d9-942a-9881780398d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d78cb72bd74e8087c6cfca67d6f263'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:26.379031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c702-495e-4282-9310-922e1bf08e17 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.384836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.385603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.385890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.386168 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.386963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.388680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.399385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1053/2095] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:26.403431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.414025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2fc2f-12ea-4d9b-9376-e6424ab216f0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.418887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281a210-1fae-41e3-9a5f-4711222d8801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.419813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281a210-1fae-41e3-9a5f-4711222d8801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.420089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281a210-1fae-41e3-9a5f-4711222d8801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.420362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281a210-1fae-41e3-9a5f-4711222d8801 admin admin] PATH_INFO: `/v3/projects/c68357d40a5b40a4b6a4546dd63dee32/users/e873230edc184017b3faee6838b28015/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.436459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281a210-1fae-41e3-9a5f-4711222d8801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68357d40a5b40a4b6a4546dd63dee32, user_id=e873230edc184017b3faee6838b28015, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.439175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281a210-1fae-41e3-9a5f-4711222d8801 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.462284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1043/2096] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:25 2022] DELETE /identity/v3/users/47d78cb72bd74e8087c6cfca67d6f263 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:26.467151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.467557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.467884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.486551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:26.487701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.498154 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:26.553340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1044/2097] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:26.562169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1054/2098] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:26 2022] PUT /identity/v3/projects/c68357d40a5b40a4b6a4546dd63dee32/users/e873230edc184017b3faee6838b28015/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:26.565423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.565778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.574044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.574335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.574617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.621160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] MFA Rules not processed for user `e873230edc184017b3faee6838b28015`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:26.623674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.652323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a7523-6222-4b58-84a0-4843ad5967ee None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.703676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1055/2099] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:26.707769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.716267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.716967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.717250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.717528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.762368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:26.780341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e1621-c19c-4d3c-a67e-2476c0f12710 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.783960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.785743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.785878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.789709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.789709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.789709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.789709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.790289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.795828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1045/2100] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:51:26.854029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.855562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:26.869579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4391455-da73-4844-8e27-d36708d324a5 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:26.876057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:26.877026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:26.877313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:26.877638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] PATH_INFO: `/v3/projects/7cec1543909e407db0cf764480d31a32` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:26.878930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cec1543909e407db0cf764480d31a32)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:26.881004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:26.907068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:26.916545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1056/2101] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:26 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:26.918913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:26.988595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b39fba-09ca-4d77-aaaa-24bb8d744fae admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.011228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cec1543909e407db0cf764480d31a32', 'request_id': 'req-13af99be-06ef-4d5d-b141-6a16b83364f5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.065527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc734-c04c-42f6-b49f-7ccdf1f0a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.066828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc734-c04c-42f6-b49f-7ccdf1f0a046 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.067115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc734-c04c-42f6-b49f-7ccdf1f0a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.067391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc734-c04c-42f6-b49f-7ccdf1f0a046 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.068095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedc734-c04c-42f6-b49f-7ccdf1f0a046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.070834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedc734-c04c-42f6-b49f-7ccdf1f0a046 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.107612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cec1543909e407db0cf764480d31a32', 'request_id': 'req-13af99be-06ef-4d5d-b141-6a16b83364f5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.114495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cec1543909e407db0cf764480d31a32', 'request_id': 'req-13af99be-06ef-4d5d-b141-6a16b83364f5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.127403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] The token cache is being invalidate because project 7cec1543909e407db0cf764480d31a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:27.127697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.128416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-13af99be-06ef-4d5d-b141-6a16b83364f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.130488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1046/2102] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:26 2022] DELETE /identity/v3/projects/7cec1543909e407db0cf764480d31a32 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:27.137166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.137166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.137840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.156950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:27.158211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.166943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1047/2103] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:27.172119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1057/2104] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:26 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:27.177187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.178688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.182746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.183040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.183317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.200144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2532533-925f-4fa2-b13d-311cd58c9b49 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.204303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:27.205535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.205678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.206222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.206609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.206892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.207740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.209936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.214866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1058/2105] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:27.219373 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.222167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1048/2106] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:27.226033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.228736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f0c0d-2ae9-43b9-9e32-4bfbf359ad0b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.237230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece9e5c-e8a7-4bed-96ab-f1070f3ea9fb admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.241831 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.242599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.243292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.243292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.243475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.243765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.244060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:27.244172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.244369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.244575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] PATH_INFO: `/v3/projects/4747ff95f5124a9ca3598592d70aedcc/users/ac67c992bee8433cb938be382dd575ed/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.246018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.260752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1059/2107] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:51:27.261998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.263979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4747ff95f5124a9ca3598592d70aedcc, user_id=ac67c992bee8433cb938be382dd575ed, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.264335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.267016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.278539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e72daf-bc0a-4105-8b4a-daaa3bb845fa admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.284899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.285999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.286232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.286509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] PATH_INFO: `/v3/projects/6e9a5f8224b440c6b1b9e9a6b366fae7` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.287737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9a5f8224b440c6b1b9e9a6b366fae7)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.289851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.385440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1049/2108] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:27 2022] PUT /identity/v3/projects/4747ff95f5124a9ca3598592d70aedcc/users/ac67c992bee8433cb938be382dd575ed/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:27.385440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.417185 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:27.423861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9a5f8224b440c6b1b9e9a6b366fae7', 'request_id': 'req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.429863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9a5f8224b440c6b1b9e9a6b366fae7', 'request_id': 'req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.447621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133b65-a78e-462b-95f0-5fe1766754c0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.496417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9a5f8224b440c6b1b9e9a6b366fae7', 'request_id': 'req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.504002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.504700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.505003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.505304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.506916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.509645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.509968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] The token cache is being invalidate because project 6e9a5f8224b440c6b1b9e9a6b366fae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:27.510258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.510912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.512576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1060/2109] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:27 2022] DELETE /identity/v3/projects/6e9a5f8224b440c6b1b9e9a6b366fae7 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:27.513527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.514910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.527070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:27.528147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5f45c-ec8c-424c-8833-a150bd0dfcc7 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.532834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.533497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.534325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.534325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.534835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.536563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.547666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1061/2110] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:27.548673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.550357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.564566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd2c52-20b8-4408-8700-0712fe8b8d2c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.569211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.570767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.571050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.571325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] PATH_INFO: `/v3/users/f705d5aa8523493899c670a524b7fe37` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.572570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f705d5aa8523493899c670a524b7fe37)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.574541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.598649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1050/2111] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:27 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:27.598649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.657131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7575b-b5f8-479e-9543-3fc3d5d8b9a0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.662168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.663060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.663867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.664318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] PATH_INFO: `/v3/projects/4747ff95f5124a9ca3598592d70aedcc/users/ac67c992bee8433cb938be382dd575ed/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.666370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4747ff95f5124a9ca3598592d70aedcc, user_id=ac67c992bee8433cb938be382dd575ed, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.669124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.680120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1051/2112] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:27 2022] PUT /identity/v3/projects/4747ff95f5124a9ca3598592d70aedcc/users/ac67c992bee8433cb938be382dd575ed/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:27.683886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.685851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.751031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915bac3-ef24-4afe-92d8-485d5b844f93 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.755680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148132f0-2c7f-4214-b41d-7afcad8eb370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.756347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148132f0-2c7f-4214-b41d-7afcad8eb370 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.756677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148132f0-2c7f-4214-b41d-7afcad8eb370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.756961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148132f0-2c7f-4214-b41d-7afcad8eb370 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.757692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148132f0-2c7f-4214-b41d-7afcad8eb370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.759599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148132f0-2c7f-4214-b41d-7afcad8eb370 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.813413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f705d5aa8523493899c670a524b7fe37'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.813994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f705d5aa8523493899c670a524b7fe37'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.820879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f705d5aa8523493899c670a524b7fe37'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.821514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f705d5aa8523493899c670a524b7fe37'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.848040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1052/2113] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:27 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:27.848040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.848040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.848040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.873305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] MFA Rules not processed for user `ac67c992bee8433cb938be382dd575ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:27.874512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.879059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f705d5aa8523493899c670a524b7fe37'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:27.898205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1062/2114] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:27 2022] DELETE /identity/v3/users/f705d5aa8523493899c670a524b7fe37 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:27.899315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.900842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.958312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1053/2115] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:27.972099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2916a41-3146-425b-93bc-156b9dd5f89b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:27.972484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:27.974306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:27.978884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:27.979663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:27.980040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:27.980440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:27.981267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:27.983898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:27.997627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1063/2116] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:28.001500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:28.012395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee51a-20df-4fc1-a009-488e93548384 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.017410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.018117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.018453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.018735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.019465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.021199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:28.032859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1064/2117] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:28.037132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:28.041584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8b5f2-342a-4e46-9808-865ef7724d8a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.047374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac87fda-e401-41ee-8385-791ce0d5a8a2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.052091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2904ce2-8822-4589-8430-7425c63562a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.053136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2904ce2-8822-4589-8430-7425c63562a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.053420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2904ce2-8822-4589-8430-7425c63562a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.053761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2904ce2-8822-4589-8430-7425c63562a9 admin admin] PATH_INFO: `/v3/projects/d8a270d9d39c4b8d916906ec77e5dfdf/users/41d8a0c9dc324b199ad2fd3c0dc8d2ba/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.067780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:28.073111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2904ce2-8822-4589-8430-7425c63562a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a270d9d39c4b8d916906ec77e5dfdf, user_id=41d8a0c9dc324b199ad2fd3c0dc8d2ba, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.076111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2904ce2-8822-4589-8430-7425c63562a9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:28.085760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:28.098775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.099865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.100161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.100449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.106212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:28.152555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1065/2118] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:28 2022] PUT /identity/v3/projects/d8a270d9d39c4b8d916906ec77e5dfdf/users/41d8a0c9dc324b199ad2fd3c0dc8d2ba/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:28.161493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.161923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.162307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.192237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] MFA Rules not processed for user `41d8a0c9dc324b199ad2fd3c0dc8d2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:28.193812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:28.202780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.205560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:28.218718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1054/2119] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:51:28.290242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1066/2120] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:28.302526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:28.304997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:28.370784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6194ab1c-7dfd-47bf-9275-f7b25fb00d63 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.377318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:28.377318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:28.442463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c27fa-cf9e-4304-9c3e-bff35a0f7dde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.446546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c27fa-cf9e-4304-9c3e-bff35a0f7dde service glance] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.446845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:28.447535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c27fa-cf9e-4304-9c3e-bff35a0f7dde service glance] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.447839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c27fa-cf9e-4304-9c3e-bff35a0f7dde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.465679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c27fa-cf9e-4304-9c3e-bff35a0f7dde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.467157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c27fa-cf9e-4304-9c3e-bff35a0f7dde service glance] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:28.472401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68bf7a-a84f-41dd-bdf1-6437de5c4833 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.478801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.480069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.480481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.480869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] PATH_INFO: `/v3/projects/2bfa456a860447289ba58defabf8003d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.482480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bfa456a860447289ba58defabf8003d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.485134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1055/2121] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:28.485304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:28.619505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bfa456a860447289ba58defabf8003d', 'request_id': 'req-a4fe092d-3699-4cab-820f-02a265bbf09a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:28.625324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bfa456a860447289ba58defabf8003d', 'request_id': 'req-a4fe092d-3699-4cab-820f-02a265bbf09a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:28.730644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bfa456a860447289ba58defabf8003d', 'request_id': 'req-a4fe092d-3699-4cab-820f-02a265bbf09a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:28.745033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] The token cache is being invalidate because project 2bfa456a860447289ba58defabf8003d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:28.745324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:28.746000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a4fe092d-3699-4cab-820f-02a265bbf09a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:28.748002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1067/2122] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:28 2022] DELETE /identity/v3/projects/2bfa456a860447289ba58defabf8003d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:28.752872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.759772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.759772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.772680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:28.774909 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:28.841668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1056/2123] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:28.849120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.849656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.849656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.870745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:28.871994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:28.880385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1068/2124] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:28.885270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:28.895087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ae104-ac78-4bbc-b07b-5e4fdb6ed78d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.901018 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.902022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.902022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.902290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.903032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:28.903344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.904984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:28.918267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1057/2125] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:51:28.922994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:28.927759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:28.936842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376753d5-2935-4d43-9148-aa6710330bdd None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:28.942937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:28.943589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:28.943877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:28.944155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:28.945004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:28.947009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:29.074890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1069/2126] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:28 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:29.086276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:29.097818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b009ad-a201-4279-8f3e-9237a06d247f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:29.161249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:29.161921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.162261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.162557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.163343 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:29.165867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:29.292146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1058/2127] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:29 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:29.296538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:29.307546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96232f1b-2d36-45f1-9cb8-ef008bfec192 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:29.312149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:29.312957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.313141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.313417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.314151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:29.315769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:29.343581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1070/2128] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:29.347589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:29.358431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f988b-4122-46b3-9ba7-6ffd86384189 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:29.364833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b41a5c-2af3-4ae8-978f-dd2af93fa379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:29.366600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b41a5c-2af3-4ae8-978f-dd2af93fa379 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.366600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b41a5c-2af3-4ae8-978f-dd2af93fa379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.366600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b41a5c-2af3-4ae8-978f-dd2af93fa379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.367007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b41a5c-2af3-4ae8-978f-dd2af93fa379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:29.368686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b41a5c-2af3-4ae8-978f-dd2af93fa379 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:29.383116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1059/2129] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:29.384259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:29.394677 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc6e86-343d-4ceb-a63c-88bd8f986b4d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:29.400250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:29.401236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.401519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.401796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] PATH_INFO: `/v3/projects/a1bb39b83c9d45d68cc1a588d44b4d04/users/98575c73aa4a49368db600bb3f7da049/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.419017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bb39b83c9d45d68cc1a588d44b4d04, user_id=98575c73aa4a49368db600bb3f7da049, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:29.421986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:29.488580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1071/2130] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:29 2022] PUT /identity/v3/projects/a1bb39b83c9d45d68cc1a588d44b4d04/users/98575c73aa4a49368db600bb3f7da049/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:29.498349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.498745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.499096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.533899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] MFA Rules not processed for user `98575c73aa4a49368db600bb3f7da049`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:29.535157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:29.567763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:29.607229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1060/2131] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:29.611895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:29.676947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130b18a-641b-4e71-b320-f5edd0ffa27f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:29.682463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:29.683157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.683437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.683800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.684449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:29.686380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:29.830361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1072/2132] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:29 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:29.834537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:29.845162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0fcebf-d703-45ef-91d2-e29755d3b8e4 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:29.902608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:29.903278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:29.903560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:29.903836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:29.904538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:29.906459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.016693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1061/2133] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:29 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:30.020649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.031483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d7d60-b8b5-4f24-8afe-f2d5854b6262 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.036117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.036890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.037223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.037443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.038197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.040301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.055834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1073/2134] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:30.057016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.067104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162db5c-a6b1-4b5c-9193-268fe2595aec admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.072709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1b19c-a0ef-4683-8f33-3264d0f5ef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.073536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1b19c-a0ef-4683-8f33-3264d0f5ef25 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.073825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1b19c-a0ef-4683-8f33-3264d0f5ef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.074123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1b19c-a0ef-4683-8f33-3264d0f5ef25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.074889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1b19c-a0ef-4683-8f33-3264d0f5ef25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.076522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1b19c-a0ef-4683-8f33-3264d0f5ef25 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.087543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1062/2135] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:30.091975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.104653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bedc7-db6b-4c81-ae46-e485e3485a02 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.109650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.110532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.110832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.111096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] PATH_INFO: `/v3/projects/5e5a696cfb4a4ac2b281cccd76495fa4/users/79721413191c4ce49454e696966de05f/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.128166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5a696cfb4a4ac2b281cccd76495fa4, user_id=79721413191c4ce49454e696966de05f, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.131021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.189522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1074/2136] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:30 2022] PUT /identity/v3/projects/5e5a696cfb4a4ac2b281cccd76495fa4/users/79721413191c4ce49454e696966de05f/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:30.201279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.201579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.201877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.239128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] MFA Rules not processed for user `79721413191c4ce49454e696966de05f`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:30.240415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:30.309532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1063/2137] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:30.313999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.373099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:30.378127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e77aa2-8fb3-42ca-8ede-17fe1f286b8a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.382753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.383410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.383695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.383972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.384712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.386712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.481971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1075/2138] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:30 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:30.486555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.498191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dec9a1-4e22-48e6-ad59-8e9694f81150 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.541762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:30.567360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.568067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.568350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.568665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.569388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.572078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.652138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1064/2139] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:30 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:30.656058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.668215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca45f22-a637-4025-a3b2-8acbb6003f11 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.673327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.674024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.674332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.674614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.675377 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.677117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.689612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1076/2140] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:30.694639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.705738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a9889-26bc-4f71-a750-80aadbe42523 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.711599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.712429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.712768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.713053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] PATH_INFO: `/v3/projects/48de39a89ec4423cb660b67b25d86e3f/users/84c0a59f755545ccb493721084bb43cc/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.730530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48de39a89ec4423cb660b67b25d86e3f, user_id=84c0a59f755545ccb493721084bb43cc, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.736731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.799961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1065/2141] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:30 2022] PUT /identity/v3/projects/48de39a89ec4423cb660b67b25d86e3f/users/84c0a59f755545ccb493721084bb43cc/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:30.804905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.815634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b3be2-736b-4329-902b-a04d3b29b4fa admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.889380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296f8f3-070f-4053-8ea1-2d77dabbd999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.890380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296f8f3-070f-4053-8ea1-2d77dabbd999 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.890670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296f8f3-070f-4053-8ea1-2d77dabbd999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.890949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296f8f3-070f-4053-8ea1-2d77dabbd999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.891676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296f8f3-070f-4053-8ea1-2d77dabbd999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.893319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296f8f3-070f-4053-8ea1-2d77dabbd999 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.903993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1077/2142] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:30.908372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:30.919881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfa694-ecd2-48b4-8494-3096a9d8a344 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:30.924662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:30.925758 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.926080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.926358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] PATH_INFO: `/v3/projects/48de39a89ec4423cb660b67b25d86e3f/users/84c0a59f755545ccb493721084bb43cc/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.928375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=48de39a89ec4423cb660b67b25d86e3f, user_id=84c0a59f755545ccb493721084bb43cc, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:30.931618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:30.943568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1066/2143] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:30 2022] PUT /identity/v3/projects/48de39a89ec4423cb660b67b25d86e3f/users/84c0a59f755545ccb493721084bb43cc/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:30.955556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea92f48-df99-49ab-8550-330f5783e514 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:30.955868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea92f48-df99-49ab-8550-330f5783e514 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:30.956122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea92f48-df99-49ab-8550-330f5783e514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:30.989290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-eea92f48-df99-49ab-8550-330f5783e514 None None] MFA Rules not processed for user `84c0a59f755545ccb493721084bb43cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:30.990426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-eea92f48-df99-49ab-8550-330f5783e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:31.079387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1078/2144] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:31.091331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:31.093431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:31.204757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d5558-89ec-43f2-8f1f-692573a4f36b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:31.210166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:31.211358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:31.211650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:31.211934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:31.278197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:31.279633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:31.289447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1067/2145] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:33.199512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:33.199989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:33.200075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:33.217857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] MFA Rules not processed for user `76fb4368530e4954b3e02e98da278589`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:33.221595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:33.266206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:33.277264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1079/2146] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:34.487919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:34.489370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:34.501062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a857b-da8e-4d38-b2f1-6541253221f0 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:34.505109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:34.505775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:34.506148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:34.506439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:34.533445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:34.571120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:34.572594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:34.581945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1068/2147] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:36.280457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:36.297321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030da451-7bb4-429a-b60c-44e180a57e44 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:36.303179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:36.304751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:36.306637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:36.307479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:36.319301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:36.322486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:36.339035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1080/2148] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:41.163921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:41.173625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19502c3-af23-48f0-a6e1-6a624148cfbe service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:41.177608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:41.178259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:41.178541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:41.178820 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:41.181477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:41.249124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:41.250566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a8e55-2379-4281-ab20-2851fd9cb4df service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:41.260799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1069/2149] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:46.329804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=8 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:50.933716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:51.010856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474dd18-9649-430f-aa76-b6e5ad3af254 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:51.016009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:51.017097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:51.017402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:51.017676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] PATH_INFO: `/v3/users/98575c73aa4a49368db600bb3f7da049` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:51.019003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98575c73aa4a49368db600bb3f7da049)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:51.028715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:51.194922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98575c73aa4a49368db600bb3f7da049'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:51.195518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98575c73aa4a49368db600bb3f7da049'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:51.203372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98575c73aa4a49368db600bb3f7da049'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:51.204012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98575c73aa4a49368db600bb3f7da049'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:51.255897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=47 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:51.290422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98575c73aa4a49368db600bb3f7da049'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:51.309525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1081/2150] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:50 2022] DELETE /identity/v3/users/98575c73aa4a49368db600bb3f7da049 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:51.314093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:51.314391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:51.314672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:51.332958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:51.334457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:51.394851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1070/2151] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:51.404861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:51.406268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:51.464136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c1e4-7b05-4ea3-84d2-c6a9aa345601 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:51.467469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:51.468118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:51.468401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:51.468752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:51.478500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:51.480682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:51.490948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1082/2152] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:51.954671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:51.956491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:51.970612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b965d96-f562-4ec5-aa30-6a541cf626c3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:51.975565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:51.976618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:51.976913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:51.977269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] PATH_INFO: `/v3/projects/a1bb39b83c9d45d68cc1a588d44b4d04` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:51.978694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1bb39b83c9d45d68cc1a588d44b4d04)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:51.982306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:52.125893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1bb39b83c9d45d68cc1a588d44b4d04', 'request_id': 'req-4618dd85-0795-44ca-9b41-e9a4c2445bec'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.210327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1bb39b83c9d45d68cc1a588d44b4d04', 'request_id': 'req-4618dd85-0795-44ca-9b41-e9a4c2445bec'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.218184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1bb39b83c9d45d68cc1a588d44b4d04', 'request_id': 'req-4618dd85-0795-44ca-9b41-e9a4c2445bec'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.228803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] The token cache is being invalidate because project a1bb39b83c9d45d68cc1a588d44b4d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:52.229722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.230698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.232306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1071/2153] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:51 2022] DELETE /identity/v3/projects/a1bb39b83c9d45d68cc1a588d44b4d04 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:52.236460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:52.246556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cf6d6-b7ae-42ef-a7f9-67c448df3442 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:52.264689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:52.307891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:52.308400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:52.308789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:52.309073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] PATH_INFO: `/v3/users/79721413191c4ce49454e696966de05f` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:52.310329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79721413191c4ce49454e696966de05f)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:52.312285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:52.562387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79721413191c4ce49454e696966de05f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.563116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79721413191c4ce49454e696966de05f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.570210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79721413191c4ce49454e696966de05f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.570921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79721413191c4ce49454e696966de05f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.706425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79721413191c4ce49454e696966de05f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:52.726708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1083/2154] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:52 2022] DELETE /identity/v3/users/79721413191c4ce49454e696966de05f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:53.195545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:53.197350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:53.211335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618dd85-0795-44ca-9b41-e9a4c2445bec admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:53.280403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:53.281381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:53.281778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:53.282044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] PATH_INFO: `/v3/projects/5e5a696cfb4a4ac2b281cccd76495fa4` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:53.283329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5a696cfb4a4ac2b281cccd76495fa4)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:53.285324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:53.389192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:53.406603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5a696cfb4a4ac2b281cccd76495fa4', 'request_id': 'req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.491761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5a696cfb4a4ac2b281cccd76495fa4', 'request_id': 'req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.501517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5a696cfb4a4ac2b281cccd76495fa4', 'request_id': 'req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.512759 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] The token cache is being invalidate because project 5e5a696cfb4a4ac2b281cccd76495fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:53.513048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.514420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.515524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1072/2155] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:53 2022] DELETE /identity/v3/projects/5e5a696cfb4a4ac2b281cccd76495fa4 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:53.519844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:53.521392 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:53.535674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b28efa-44b7-4817-86c9-7d469f9603f4 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:53.598403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:53.599351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:53.599630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:53.599903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] PATH_INFO: `/v3/users/84c0a59f755545ccb493721084bb43cc` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:53.601258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c0a59f755545ccb493721084bb43cc)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:53.603278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:53.810475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:53.813231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c0a59f755545ccb493721084bb43cc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.813889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c0a59f755545ccb493721084bb43cc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.820928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c0a59f755545ccb493721084bb43cc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.821569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c0a59f755545ccb493721084bb43cc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.924432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c0a59f755545ccb493721084bb43cc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:53.943874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1084/2156] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:53 2022] DELETE /identity/v3/users/84c0a59f755545ccb493721084bb43cc => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:54.434893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:54.438195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:54.451010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfdc80-ad06-426c-b38b-c1fa4aa592f0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:54.512618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:54.514024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:54.514024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:54.514221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] PATH_INFO: `/v3/projects/48de39a89ec4423cb660b67b25d86e3f` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:54.515585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48de39a89ec4423cb660b67b25d86e3f)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:54.524940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:54.688555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48de39a89ec4423cb660b67b25d86e3f', 'request_id': 'req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:54.690410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:54.692051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:54.706290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae15b4-d90e-4dac-9da9-4b80bb73b871 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:54.759890 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48de39a89ec4423cb660b67b25d86e3f', 'request_id': 'req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:54.767639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:54.768550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:54.768891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:54.769214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] PATH_INFO: `/v3/users/e873230edc184017b3faee6838b28015` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:54.770430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e873230edc184017b3faee6838b28015)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:54.772421 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:54.772567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48de39a89ec4423cb660b67b25d86e3f', 'request_id': 'req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:54.785710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] The token cache is being invalidate because project 48de39a89ec4423cb660b67b25d86e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:54.786010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:54.786880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bab1bd32-06ca-412a-a7f8-c7c7fd3a02e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:54.788457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1073/2157] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:54 2022] DELETE /identity/v3/projects/48de39a89ec4423cb660b67b25d86e3f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:54.793248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7daa2-c047-4c2d-a98f-54e9268162e1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:54.794255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7daa2-c047-4c2d-a98f-54e9268162e1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:54.794612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7daa2-c047-4c2d-a98f-54e9268162e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:54.813255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-62b7daa2-c047-4c2d-a98f-54e9268162e1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:54.814465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-62b7daa2-c047-4c2d-a98f-54e9268162e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:54.825246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1074/2158] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:54.833020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:54.833599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:54.833895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:54.852774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:54.854024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:54.862970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1075/2159] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:54.867491 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:54.881112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b877d-5991-4e92-b8b2-d68cc0c4856e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:54.887639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:54.888409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:54.888409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:54.888824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:54.889516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:54.889896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:54.892117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:54.907279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1076/2160] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:51:54.912020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:54.924012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c0596-44e4-4010-a19c-ce77c98e7c67 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:54.927284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:54.927933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:54.928227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:54.928589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:54.932951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:54.937422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:54.955853 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:55.255838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1077/2161] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:54 2022] POST /identity/v3/projects => generated 385 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:55.260211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.261871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e873230edc184017b3faee6838b28015'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:55.263391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e873230edc184017b3faee6838b28015'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:55.274897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e873230edc184017b3faee6838b28015'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:55.282575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e873230edc184017b3faee6838b28015'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:55.338033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aff5e-a326-4792-a63d-4f90ae0682b0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:55.344157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:55.344907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.345238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.345524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.346235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:55.348192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:55.359051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f76526a1-7f53-4c0f-a9f4-eb042389a170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e873230edc184017b3faee6838b28015'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:55.379925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1085/2162] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:54 2022] DELETE /identity/v3/users/e873230edc184017b3faee6838b28015 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:55.384596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11806492-d8a3-4652-a770-98df635388b1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.384888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11806492-d8a3-4652-a770-98df635388b1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.385167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11806492-d8a3-4652-a770-98df635388b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.403507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-11806492-d8a3-4652-a770-98df635388b1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:55.404738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-11806492-d8a3-4652-a770-98df635388b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:55.439808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:55.450899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1078/2163] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:55 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:55.455075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.460288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:55.467567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1086/2164] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:55.471162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15174930-3233-4a4c-89f2-76a79189b8d1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:55.475775 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:55.476511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.476813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.477142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.477952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:55.479353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11806492-d8a3-4652-a770-98df635388b1 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.479723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:55.481476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-11806492-d8a3-4652-a770-98df635388b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:55.493555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1079/2165] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:55.499149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.505539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aa9d3-31f9-44a7-9510-1139b81a7100 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:55.510385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:55.511134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.511349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.511634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.512426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:55.516652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:55.524902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1080/2166] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:55.529107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.540256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8d41e-ef7f-4a2a-9063-18fde929cc53 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:55.541441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11806492-d8a3-4652-a770-98df635388b1 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:55.546153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:55.546583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325a8a3-4d4a-49d5-a2e3-3a12e6a09e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:55.547050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.547344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.547496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325a8a3-4d4a-49d5-a2e3-3a12e6a09e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.547610 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] PATH_INFO: `/v3/projects/e286fb2098774a23b107a0fcefd41dd8/users/43e41e332091480488411308c9737bd9/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.547730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325a8a3-4d4a-49d5-a2e3-3a12e6a09e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.547920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325a8a3-4d4a-49d5-a2e3-3a12e6a09e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.557643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325a8a3-4d4a-49d5-a2e3-3a12e6a09e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:55.559800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325a8a3-4d4a-49d5-a2e3-3a12e6a09e10 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:55.562984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e286fb2098774a23b107a0fcefd41dd8, user_id=43e41e332091480488411308c9737bd9, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:55.565717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:55.569958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1087/2167] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:51:55.668019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1081/2168] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:55 2022] PUT /identity/v3/projects/e286fb2098774a23b107a0fcefd41dd8/users/43e41e332091480488411308c9737bd9/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:55.678568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.678860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.679136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.706816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] MFA Rules not processed for user `43e41e332091480488411308c9737bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:55.709413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:55.775054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1088/2169] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:55.787615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.790030 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:55.878469 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58747c1-b150-4268-85f6-06ad03b3cdcc admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:55.883550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:55.884221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:55.884538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:55.893166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:55.894279 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:55.895724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:55.906037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1082/2170] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:55.967302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:55.969127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:56.025885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:56.028493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:56.055468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94520f80-74c8-4b8a-b7ad-84b38c89d540 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:56.066024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:56.066024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:56.066024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:56.066024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] PATH_INFO: `/v3/users/42f7a8af3b814cb592baa5a4ef5909c3` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:56.066562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f7a8af3b814cb592baa5a4ef5909c3)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:56.067212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:56.083798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:56.100981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:56.120236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eab630-4390-4fd0-b961-55c544969d99 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:56.124786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:56.125652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:56.125945 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:56.126222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] PATH_INFO: `/v3/projects/c68357d40a5b40a4b6a4546dd63dee32` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:56.127430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68357d40a5b40a4b6a4546dd63dee32)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:56.129422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:56.234196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68357d40a5b40a4b6a4546dd63dee32', 'request_id': 'req-6dc0ffbb-704d-45eb-932c-d9390bf59907'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.301417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68357d40a5b40a4b6a4546dd63dee32', 'request_id': 'req-6dc0ffbb-704d-45eb-932c-d9390bf59907'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.308373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68357d40a5b40a4b6a4546dd63dee32', 'request_id': 'req-6dc0ffbb-704d-45eb-932c-d9390bf59907'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.315721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f7a8af3b814cb592baa5a4ef5909c3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.316389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f7a8af3b814cb592baa5a4ef5909c3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.319719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] The token cache is being invalidate because project c68357d40a5b40a4b6a4546dd63dee32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:56.320036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.320729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.322485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1083/2171] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:56 2022] DELETE /identity/v3/projects/c68357d40a5b40a4b6a4546dd63dee32 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:56.323807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f7a8af3b814cb592baa5a4ef5909c3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.325061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f7a8af3b814cb592baa5a4ef5909c3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.326787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:56.328204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:56.398714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0ffbb-704d-45eb-932c-d9390bf59907 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:56.404066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:56.405040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:56.405330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:56.405609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] PATH_INFO: `/v3/users/ac67c992bee8433cb938be382dd575ed` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:56.406864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac67c992bee8433cb938be382dd575ed)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:56.408852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:56.415606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-dd9f0e53-e080-4afc-b019-159bf9317266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f7a8af3b814cb592baa5a4ef5909c3'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.449291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1089/2172] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:55 2022] DELETE /identity/v3/users/42f7a8af3b814cb592baa5a4ef5909c3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:56.459847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:56.459847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:56.459847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:56.476477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:56.477934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:56.552808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1090/2173] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:56.559872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:56.561364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:56.620359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2a3b9-9675-4f9e-8495-bf88e12f7ecf None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:56.624663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:56.625958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:56.625958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:56.625958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:56.635459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:56.637621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:56.647079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1091/2174] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:51:56.740326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac67c992bee8433cb938be382dd575ed'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.741192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac67c992bee8433cb938be382dd575ed'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.741886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac67c992bee8433cb938be382dd575ed'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.755374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac67c992bee8433cb938be382dd575ed'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.762964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac67c992bee8433cb938be382dd575ed'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:56.842650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:56.847957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1084/2175] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:51:56 2022] DELETE /identity/v3/users/ac67c992bee8433cb938be382dd575ed => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:57.096488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.098850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.114717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcaa21-eecb-40dd-9ba7-b5dca30f6af9 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.158509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:57.185152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.186617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.186617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.186819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] PATH_INFO: `/v3/projects/31a8d370fefa40278d50e044e3d7d233` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.187939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a8d370fefa40278d50e044e3d7d233)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.189924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.295540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a8d370fefa40278d50e044e3d7d233', 'request_id': 'req-5e04517c-2181-4dfe-b2a0-87c64be73458'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.303337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a8d370fefa40278d50e044e3d7d233', 'request_id': 'req-5e04517c-2181-4dfe-b2a0-87c64be73458'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.310938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.312685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.333398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce4a47-2c81-4d27-8f80-20226247eaa2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.393591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a8d370fefa40278d50e044e3d7d233', 'request_id': 'req-5e04517c-2181-4dfe-b2a0-87c64be73458'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.394515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.395389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.395679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.395957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] PATH_INFO: `/v3/projects/4747ff95f5124a9ca3598592d70aedcc` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.397247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4747ff95f5124a9ca3598592d70aedcc)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.399315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.406853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] The token cache is being invalidate because project 31a8d370fefa40278d50e044e3d7d233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:57.407149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.407812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5e04517c-2181-4dfe-b2a0-87c64be73458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.409511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1092/2176] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:57 2022] DELETE /identity/v3/projects/31a8d370fefa40278d50e044e3d7d233 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:57.414183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691166-0b85-482c-95f2-4a5ec2a5b61f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.415788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691166-0b85-482c-95f2-4a5ec2a5b61f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.416071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691166-0b85-482c-95f2-4a5ec2a5b61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.434978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ef691166-0b85-482c-95f2-4a5ec2a5b61f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:57.436207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ef691166-0b85-482c-95f2-4a5ec2a5b61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.444187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1093/2177] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:57.452160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b15486-3745-4f15-a233-a0b467663400 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.452456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b15486-3745-4f15-a233-a0b467663400 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.452776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b15486-3745-4f15-a233-a0b467663400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.471490 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-19b15486-3745-4f15-a233-a0b467663400 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:57.472728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-19b15486-3745-4f15-a233-a0b467663400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.481856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1094/2178] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:57.489602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b15486-3745-4f15-a233-a0b467663400 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.494527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4747ff95f5124a9ca3598592d70aedcc', 'request_id': 'req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.496297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b15486-3745-4f15-a233-a0b467663400 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.560737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.561407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.561689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.562012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.562808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:57.563109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.564859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.570252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4747ff95f5124a9ca3598592d70aedcc', 'request_id': 'req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.578153 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4747ff95f5124a9ca3598592d70aedcc', 'request_id': 'req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.582137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1095/2179] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:51:57.583784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.594882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] The token cache is being invalidate because project 4747ff95f5124a9ca3598592d70aedcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:51:57.595394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.596548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b5a1b6fb-5971-48b3-a0ce-86d155a9618e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:51:57.597155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a325dff-aeb0-4535-a1e5-0eb3a6f2e5f1 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.598222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1085/2180] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:51:57 2022] DELETE /identity/v3/projects/4747ff95f5124a9ca3598592d70aedcc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:57.604352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fa4b6-be8d-4a32-b1c5-7a80f667b0a8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.604677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fa4b6-be8d-4a32-b1c5-7a80f667b0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.604967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fa4b6-be8d-4a32-b1c5-7a80f667b0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.607263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.607994 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.608282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.608597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.609464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.616287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.625071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-254fa4b6-be8d-4a32-b1c5-7a80f667b0a8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:57.626525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-254fa4b6-be8d-4a32-b1c5-7a80f667b0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.643686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1086/2181] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:57.650348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.650693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.650977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.673291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:57.676324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.684155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1087/2182] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:57.688298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.705704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404af9bf-1cd7-4963-929e-3f23974709bc None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.711289 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.711965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.712253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.712562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.713304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:51:57.713619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.716573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.729226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1088/2183] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:51:57.733778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.744029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1096/2184] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:57.745916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fa61a-9c21-4426-a7f1-1832e5bf34f9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.751753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.753954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:57.789526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:57.803608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:57.811799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.812471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.812854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.813136 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.815335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.817355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.833358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e4ee8-ed63-46b1-83c1-5ce8162b68cb admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.838358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.839036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.839317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.839592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.840946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.846030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.902214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1089/2185] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:57.907053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.924268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1097/2186] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:57.929118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:57.939461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225f519-8fba-4ad1-bb0f-e6637829576b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.969794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd16520-f0f3-4c89-8d51-5053a48114c8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:57.974839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.975633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.976031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.976446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:57.977188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:57.979178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:57.998462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:57.999130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:57.999415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:57.999687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.000404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.002027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.011537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1098/2187] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:58.016026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.025209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bc1db-9cf2-4c08-89d1-58aaedce1cc2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.029771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.030388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.030668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.030938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.031690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.033292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.042700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1099/2188] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:58.046593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.055713 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938c0f4-e799-4812-bb45-f8153875eb05 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.060998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f85530-04fc-4ea9-8812-46cb93b53eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.061845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85530-04fc-4ea9-8812-46cb93b53eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.062128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85530-04fc-4ea9-8812-46cb93b53eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.062401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85530-04fc-4ea9-8812-46cb93b53eb7 admin admin] PATH_INFO: `/v3/projects/2f97b57909174abeab16dedc077b37e0/users/4b019c1d50bc4772b8a60ca73ab78dd0/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.078578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85530-04fc-4ea9-8812-46cb93b53eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f97b57909174abeab16dedc077b37e0, user_id=4b019c1d50bc4772b8a60ca73ab78dd0, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.081293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85530-04fc-4ea9-8812-46cb93b53eb7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.096595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1090/2189] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:57 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:51:58.100582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.110418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646b654-2414-4f60-ac24-ca347d9737e8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.115778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.116457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.116774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.117092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.117845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.119475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.130726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1091/2190] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:58.134639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.145735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57a99-d90a-4805-9733-878d9094964d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.150654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.151323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.151701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.151982 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.153342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.154952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.170889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1092/2191] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:51:58.171082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1100/2192] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:58 2022] PUT /identity/v3/projects/2f97b57909174abeab16dedc077b37e0/users/4b019c1d50bc4772b8a60ca73ab78dd0/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:58.172222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.179631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.179927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.180282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.182206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f3cad-be70-46ae-b0bc-b6f0fe98773f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.199057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:58.201926 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:58.240035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d11673-5915-4b53-a3f3-4034119d1b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.240967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11673-5915-4b53-a3f3-4034119d1b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.241257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11673-5915-4b53-a3f3-4034119d1b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.241545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d11673-5915-4b53-a3f3-4034119d1b64 admin admin] PATH_INFO: `/v3/projects/65b6d1d933c5490da04347641210dfae/users/94148b3db852495eabd9d19824852ea4/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.258294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11673-5915-4b53-a3f3-4034119d1b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b6d1d933c5490da04347641210dfae, user_id=94148b3db852495eabd9d19824852ea4, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.261276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d11673-5915-4b53-a3f3-4034119d1b64 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.267024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1101/2193] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:58.278411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.282688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:58.339097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f117-da70-4118-aef2-1240dacdffb6 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.342832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.343467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.343732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.344665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.354150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.356077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.366100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1093/2194] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:58 2022] PUT /identity/v3/projects/65b6d1d933c5490da04347641210dfae/users/94148b3db852495eabd9d19824852ea4/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:51:58.366386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1102/2195] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:58.378019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.378320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.378603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.408771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] MFA Rules not processed for user `94148b3db852495eabd9d19824852ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:58.410090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:58.478766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1094/2196] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:58.483793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.493702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728c174-638d-4abd-be58-6f533683e721 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.553949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.554627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.554916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.555195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.555894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.557915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.663676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1103/2197] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:51:58 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:58.664979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.679794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b579a58-5be7-4838-af3e-2fe2e0b4d1bf None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.746830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.747672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.747941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.748217 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.748974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.749666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:58.750893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.834823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:58.849891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1095/2198] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:51:58 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:51:58.854440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.867635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4f3fe-af77-4725-b550-cf8bc21b77cd admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.875278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.876405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.876836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.877123 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.877928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.882197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:58.901627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1104/2199] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:58.903071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:58.916720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b7eb4-7c03-432a-8c79-04bcb28bed91 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:58.928268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:58.929287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:58.929521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:58.929983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] PATH_INFO: `/v3/projects/05068dde0c93439ab4d602fb719bb4a5/users/a26ef6da26644b8fa47621bc5dce61e0/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:58.960257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05068dde0c93439ab4d602fb719bb4a5, user_id=a26ef6da26644b8fa47621bc5dce61e0, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:58.964532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:59.064540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1096/2200] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:58 2022] PUT /identity/v3/projects/05068dde0c93439ab4d602fb719bb4a5/users/a26ef6da26644b8fa47621bc5dce61e0/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:59.068460 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:59.080159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf89f8a-3a6e-4112-a5fa-74d0f94c1e7b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:59.141593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd80cc1-1c60-4f92-beff-aa4597ee3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:59.142186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd80cc1-1c60-4f92-beff-aa4597ee3ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:59.142466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd80cc1-1c60-4f92-beff-aa4597ee3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:59.142739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd80cc1-1c60-4f92-beff-aa4597ee3ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:59.143472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd80cc1-1c60-4f92-beff-aa4597ee3ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:59.145130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd80cc1-1c60-4f92-beff-aa4597ee3ee5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:59.156024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1105/2201] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:51:59.160302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:59.175409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb243c-3ead-480a-91e1-2f8be271ad4a admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:59.183242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:59.191031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:59.191031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:59.191031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] PATH_INFO: `/v3/projects/05068dde0c93439ab4d602fb719bb4a5/users/a26ef6da26644b8fa47621bc5dce61e0/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:59.191031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05068dde0c93439ab4d602fb719bb4a5, user_id=a26ef6da26644b8fa47621bc5dce61e0, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:59.191031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:59.203082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1097/2202] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:51:59 2022] PUT /identity/v3/projects/05068dde0c93439ab4d602fb719bb4a5/users/a26ef6da26644b8fa47621bc5dce61e0/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:51:59.213500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:59.214096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:59.214096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:59.244728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] MFA Rules not processed for user `a26ef6da26644b8fa47621bc5dce61e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:51:59.247100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:59.326374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1106/2203] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:51:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:51:59.337953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:59.339674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:59.398768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552016d0-3806-41b1-91db-b25f25278397 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:59.403126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ec0e0-ebb0-44a4-af51-9b3d871fe18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:59.406356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ec0e0-ebb0-44a4-af51-9b3d871fe18a service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:59.406356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ec0e0-ebb0-44a4-af51-9b3d871fe18a service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:59.406356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ec0e0-ebb0-44a4-af51-9b3d871fe18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:59.479582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:59.497519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ec0e0-ebb0-44a4-af51-9b3d871fe18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:59.500567 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ec0e0-ebb0-44a4-af51-9b3d871fe18a service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:59.519553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1098/2204] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:51:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:59.724013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:00.330639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:00.332136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:00.431867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb967943-7541-44bb-88a1-335d5784f69f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:00.437393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:00.438299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:00.439003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:00.439003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] PATH_INFO: `/v3/users/43e41e332091480488411308c9737bd9` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:00.440032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e41e332091480488411308c9737bd9)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:00.442017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:00.699166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e41e332091480488411308c9737bd9'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:00.702502 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e41e332091480488411308c9737bd9'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:00.713950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e41e332091480488411308c9737bd9'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:00.714712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e41e332091480488411308c9737bd9'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:00.829418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e41e332091480488411308c9737bd9'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:00.854854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1107/2205] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:52:00 2022] DELETE /identity/v3/users/43e41e332091480488411308c9737bd9 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:00.860400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:00.860914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:00.861021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:00.882728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:00.883949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:00.944960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1099/2206] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:00.959402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:00.959402 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:01.013939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:01.036689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f35f16-32c3-4314-992f-a8ddaa297f91 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:01.040649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2534a78-0522-4d13-b521-b11f3addd08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:01.041469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2534a78-0522-4d13-b521-b11f3addd08a service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:01.041755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2534a78-0522-4d13-b521-b11f3addd08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:01.042026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2534a78-0522-4d13-b521-b11f3addd08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:01.052519 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2534a78-0522-4d13-b521-b11f3addd08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:01.054053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2534a78-0522-4d13-b521-b11f3addd08a service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:01.071955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1108/2207] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:01.646777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:01.648333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:01.664177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed986741-2781-4052-af4d-cc928d6160f1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:01.673048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:01.674058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:01.674351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:01.674632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] PATH_INFO: `/v3/projects/e286fb2098774a23b107a0fcefd41dd8` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:01.676148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e286fb2098774a23b107a0fcefd41dd8)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:01.678272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:01.827660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e286fb2098774a23b107a0fcefd41dd8', 'request_id': 'req-958b7448-fce3-44eb-b476-9b013e19ef7e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:01.941741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e286fb2098774a23b107a0fcefd41dd8', 'request_id': 'req-958b7448-fce3-44eb-b476-9b013e19ef7e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:01.950639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e286fb2098774a23b107a0fcefd41dd8', 'request_id': 'req-958b7448-fce3-44eb-b476-9b013e19ef7e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:01.977642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] The token cache is being invalidate because project e286fb2098774a23b107a0fcefd41dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:52:01.978143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:01.978978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-958b7448-fce3-44eb-b476-9b013e19ef7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:01.981399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1100/2208] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:52:01 2022] DELETE /identity/v3/projects/e286fb2098774a23b107a0fcefd41dd8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:52:01.986557 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:01.987041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:01.987141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.025070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:02.028078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:02.167675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1109/2209] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:02.177371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.179527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.179527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.204447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:02.206000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:02.215397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1101/2210] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:02.220628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:02.239312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fada8-14ee-44ff-90ac-4d6643ba5c4b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:02.244636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:02.247052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.247052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.247052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.247908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:52:02.248225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:02.250575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:02.267401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1110/2211] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:52:02.271072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:02.293007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de09f96-3f33-457f-bd04-c9d669b74844 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:02.300398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:02.301530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.301821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.302148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.302867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:02.305037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:02.449130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1102/2212] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:52:02 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:52:02.451664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:02.471729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc95ed-4716-404e-a826-a071595d0074 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:02.579500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:02.580313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.580638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.580917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.581718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:02.583843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:02.709226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1111/2213] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:52:02 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:52:02.710436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:02.726162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2eee8-753e-45dd-a8a0-faf805d35965 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:02.762407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:02.762968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.763257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.763545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.764297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:02.765976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:02.805629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1103/2214] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:52:02.810323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:02.839134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a51c25-1581-48e3-aafc-7f740d6ca3f4 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:02.845913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a039c-6036-497c-8a0b-a0d4682c2b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:02.847738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a039c-6036-497c-8a0b-a0d4682c2b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.848181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a039c-6036-497c-8a0b-a0d4682c2b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.848634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a039c-6036-497c-8a0b-a0d4682c2b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.849556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a039c-6036-497c-8a0b-a0d4682c2b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:02.851376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a039c-6036-497c-8a0b-a0d4682c2b18 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:02.867976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1112/2215] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:52:02.873635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:02.894042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6f4c4-3c7f-44e8-98b9-f61f51333b43 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:02.900705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:02.902915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:02.903497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:02.904437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] PATH_INFO: `/v3/projects/aa040dcc21404116bd2e8b5c782df93c/users/44e87fb6b6d44e62a73bbf490478b37e/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:02.915643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:02.925436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa040dcc21404116bd2e8b5c782df93c, user_id=44e87fb6b6d44e62a73bbf490478b37e, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:02.930752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:03.055819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1104/2216] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:52:02 2022] PUT /identity/v3/projects/aa040dcc21404116bd2e8b5c782df93c/users/44e87fb6b6d44e62a73bbf490478b37e/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:03.061676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:03.062133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:03.062357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:03.088872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:03.090741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:03.146793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:03.212086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1113/2217] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:03.221538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:03.224485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:03.298494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c71f8c-2ea8-4974-bc36-82d0d6b993b4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:03.302783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdc6ba-3b53-4551-a2c8-7f88a1f54edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:03.303446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdc6ba-3b53-4551-a2c8-7f88a1f54edb service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:03.303732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdc6ba-3b53-4551-a2c8-7f88a1f54edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:03.306920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdc6ba-3b53-4551-a2c8-7f88a1f54edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:03.316799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdc6ba-3b53-4551-a2c8-7f88a1f54edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:03.319728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdc6ba-3b53-4551-a2c8-7f88a1f54edb service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:03.333595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1105/2218] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:04.148781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:04.151082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:04.164763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8036291-d488-4c1e-807f-06abe919da4e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:04.173133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:04.174366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:04.174856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:04.175356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] PATH_INFO: `/v3/users/94148b3db852495eabd9d19824852ea4` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:04.176900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94148b3db852495eabd9d19824852ea4)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:04.179209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:04.439571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94148b3db852495eabd9d19824852ea4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:04.440408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94148b3db852495eabd9d19824852ea4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:04.452990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94148b3db852495eabd9d19824852ea4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:04.454291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94148b3db852495eabd9d19824852ea4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:04.570150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94148b3db852495eabd9d19824852ea4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:04.598034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1114/2219] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:52:04 2022] DELETE /identity/v3/users/94148b3db852495eabd9d19824852ea4 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:04.602580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467eb1ec-cf3a-4007-afb8-8a9df69220b5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:04.602580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467eb1ec-cf3a-4007-afb8-8a9df69220b5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:04.602580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467eb1ec-cf3a-4007-afb8-8a9df69220b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:04.626587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-467eb1ec-cf3a-4007-afb8-8a9df69220b5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:04.627882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-467eb1ec-cf3a-4007-afb8-8a9df69220b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:04.711384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1106/2220] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:04.734499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:04.736474 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:04.829092 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fdf4b-f30a-4f51-8550-113f73232012 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:04.836416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a53013-63b0-44f1-ab05-77ba98962a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:04.836416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53013-63b0-44f1-ab05-77ba98962a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:04.836416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53013-63b0-44f1-ab05-77ba98962a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:04.839855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53013-63b0-44f1-ab05-77ba98962a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:04.868441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a53013-63b0-44f1-ab05-77ba98962a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:04.869990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a53013-63b0-44f1-ab05-77ba98962a0a service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:04.881450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1115/2221] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:05.182435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:05.183021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:05.183953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:05.251164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] MFA Rules not processed for user `44e87fb6b6d44e62a73bbf490478b37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:05.252410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:05.381688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:05.393245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:05.393245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:05.434689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1107/2222] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:05.439962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] MFA Rules not processed for user `4b019c1d50bc4772b8a60ca73ab78dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:05.442469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:05.449770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:05.452044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:05.483713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:05.542566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:05.568275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fd5db-4485-4dfc-93d8-2ab0358b5230 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:05.581133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:05.581805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:05.582088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:05.582366 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:05.586592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1116/2223] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:05.588697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:05.595521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:05.600523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:05.600993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:05.610306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1108/2224] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:05.611387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:05.613126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:05.634925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09449d11-6626-4eb0-a896-9c9a224f2320 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:05.642731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:05.643760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:05.644159 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:05.644804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] PATH_INFO: `/v3/projects/65b6d1d933c5490da04347641210dfae` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:05.646899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b6d1d933c5490da04347641210dfae)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:05.649614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:05.656393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a008fa3-187b-43b8-b559-856cfdca6e0c service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:05.670542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:05.674859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:05.677023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:05.677492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:05.678833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:05.687658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:05.797539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1109/2225] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:52:05 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:52:05.802560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:05.892995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b6d1d933c5490da04347641210dfae', 'request_id': 'req-455726cc-25a5-4bba-993e-28bb1fec7030'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:05.901289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b6d1d933c5490da04347641210dfae', 'request_id': 'req-455726cc-25a5-4bba-993e-28bb1fec7030'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:05.904306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb120e5c-83f0-4465-b015-774fab5d838c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:05.984168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:05.984956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:05.985300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:05.985646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:05.986537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:05.995864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:06.021733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b6d1d933c5490da04347641210dfae', 'request_id': 'req-455726cc-25a5-4bba-993e-28bb1fec7030'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.036247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] The token cache is being invalidate because project 65b6d1d933c5490da04347641210dfae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:52:06.038186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.039147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.041074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1117/2226] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:52:05 2022] DELETE /identity/v3/projects/65b6d1d933c5490da04347641210dfae => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:06.045988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:06.047666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:06.062701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455726cc-25a5-4bba-993e-28bb1fec7030 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:06.067531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:06.068463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:06.069712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:06.069712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] PATH_INFO: `/v3/users/a26ef6da26644b8fa47621bc5dce61e0` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:06.070757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26ef6da26644b8fa47621bc5dce61e0)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:06.072790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:06.127435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1110/2227] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:52:05 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:52:06.129388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:06.131058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:06.143923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d891d23-cf4d-4c72-a737-66f4a75d7959 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:06.148919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:06.149588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:06.149879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:06.150162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:06.150926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:06.152804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:06.166499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1111/2228] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:52:06.167526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:06.178026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33980ee-72f6-4371-b598-d5a2897dae44 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:06.182868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:06.184435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:06.184755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:06.187204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] PATH_INFO: `/v3/projects/e4e0a97e86df4bd38461b02c6b2db08c/users/ecd46537ac5246b191b605f9cf141710/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:06.200781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e0a97e86df4bd38461b02c6b2db08c, user_id=ecd46537ac5246b191b605f9cf141710, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:06.206455 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:06.297594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1112/2229] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:52:06 2022] PUT /identity/v3/projects/e4e0a97e86df4bd38461b02c6b2db08c/users/ecd46537ac5246b191b605f9cf141710/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:06.301194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:06.312287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26ef6da26644b8fa47621bc5dce61e0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.313023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26ef6da26644b8fa47621bc5dce61e0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.321297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26ef6da26644b8fa47621bc5dce61e0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.321709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26ef6da26644b8fa47621bc5dce61e0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.369870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63734b2e-b31f-465e-a29f-6fa827506b1c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:06.375966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:06.376688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:06.376978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:06.377259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:06.378052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:06.379621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:06.391089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1113/2230] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:52:06.395204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:06.397037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:06.410757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25b4b3-9083-494e-925a-4796835b6407 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:06.415441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:06.416759 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:06.416859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:06.417146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] PATH_INFO: `/v3/projects/e4e0a97e86df4bd38461b02c6b2db08c/users/ecd46537ac5246b191b605f9cf141710/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:06.419702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29751050-44e5-4e55-b8af-8b1833ff1fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26ef6da26644b8fa47621bc5dce61e0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:06.420285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e0a97e86df4bd38461b02c6b2db08c, user_id=ecd46537ac5246b191b605f9cf141710, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:06.423190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:06.438381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1114/2231] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:52:06 2022] PUT /identity/v3/projects/e4e0a97e86df4bd38461b02c6b2db08c/users/ecd46537ac5246b191b605f9cf141710/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:52:06.446647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1118/2232] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:52:06 2022] DELETE /identity/v3/users/a26ef6da26644b8fa47621bc5dce61e0 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:06.690465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:06.953031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:06.957187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:07.068456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc00e2d-b55d-4c20-9e36-0297b35b8f8f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:07.135778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:07.136987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.137709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.138138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] PATH_INFO: `/v3/projects/05068dde0c93439ab4d602fb719bb4a5` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.139427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05068dde0c93439ab4d602fb719bb4a5)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:07.141539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:07.280362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05068dde0c93439ab4d602fb719bb4a5', 'request_id': 'req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:07.356120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05068dde0c93439ab4d602fb719bb4a5', 'request_id': 'req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:07.368539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05068dde0c93439ab4d602fb719bb4a5', 'request_id': 'req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:07.381723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] The token cache is being invalidate because project 05068dde0c93439ab4d602fb719bb4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:52:07.382152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:07.383221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9625ae67-77b6-4cfd-90d3-deecd7c14ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:07.384699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1115/2233] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:52:06 2022] DELETE /identity/v3/projects/05068dde0c93439ab4d602fb719bb4a5 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:07.391163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.391704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.392345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.419521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:07.420881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:07.471628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:07.473183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:07.495796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1119/2234] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:07.505768 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.506081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.506405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.526513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:07.528441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:07.538756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1116/2235] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:52:07.543859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:07.555484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7399dd-101d-4c01-a395-83aef99406d3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:07.561401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:07.562344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.562561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.563124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.565364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:52:07.565689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:07.567465 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:07.580341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1120/2236] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:52:07.595265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:07.622416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3829b0-a101-4205-b1fb-f3bf7f6e4f44 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:07.634500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:07.635350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.635536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.635866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.637670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:07.639686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:07.738934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1117/2237] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:52:07 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:52:07.742577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:07.752494 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b78e-3bb8-4417-afe0-ea1aeec84d96 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:07.840520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:07.841301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.841582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.841851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.842540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:07.844489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:07.943420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1121/2238] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:52:07 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:52:07.947612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:07.963202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da640bf6-24cd-4ef7-94ba-896b9a0a66d0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:07.975577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:07.977397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:07.977397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:07.977397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:07.977642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:07.979443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:07.997600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1118/2239] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:52:08.001661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:08.012976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becc92f-fa92-4145-9ed1-53e2467ba39b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:08.018471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a294b0-cb99-4753-9f4d-a912477a7496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:08.019270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a294b0-cb99-4753-9f4d-a912477a7496 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:08.019680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a294b0-cb99-4753-9f4d-a912477a7496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:08.020069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a294b0-cb99-4753-9f4d-a912477a7496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:08.020970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a294b0-cb99-4753-9f4d-a912477a7496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:08.022736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a294b0-cb99-4753-9f4d-a912477a7496 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:08.039321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1122/2240] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:52:08.040632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:08.051662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fa576-e3b0-4442-8398-4a17c6b215f6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:08.056617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:08.057520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:08.057805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:08.058084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] PATH_INFO: `/v3/projects/a518e9aade704470ad83e19420cb0f5e/users/0e53671165414fce9d81d31964db486f/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:08.083997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a518e9aade704470ad83e19420cb0f5e, user_id=0e53671165414fce9d81d31964db486f, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:08.087946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:08.210531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1119/2241] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:52:08 2022] PUT /identity/v3/projects/a518e9aade704470ad83e19420cb0f5e/users/0e53671165414fce9d81d31964db486f/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:08.221215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117444-6940-4d0d-b768-8a278de4886d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:08.221584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117444-6940-4d0d-b768-8a278de4886d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:08.221955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc117444-6940-4d0d-b768-8a278de4886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:08.249917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cc117444-6940-4d0d-b768-8a278de4886d None None] MFA Rules not processed for user `0e53671165414fce9d81d31964db486f`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:08.251088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cc117444-6940-4d0d-b768-8a278de4886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:08.327608 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1123/2242] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:08.349787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:08.352746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:08.457752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04986c89-ebc1-4d37-922a-cbfdcf743131 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:08.468361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:08.469244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:08.469477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:08.472066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:08.482618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:08.484830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:08.499104 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1120/2243] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:12.990529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a33ee-55b5-4118-a90c-ae93550da27f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:12.991018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a33ee-55b5-4118-a90c-ae93550da27f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:12.991121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a33ee-55b5-4118-a90c-ae93550da27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:13.020682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ab6a33ee-55b5-4118-a90c-ae93550da27f None None] MFA Rules not processed for user `ecd46537ac5246b191b605f9cf141710`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:13.021910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ab6a33ee-55b5-4118-a90c-ae93550da27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:13.050440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:13.094915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1124/2244] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:13.108922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:13.110465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:13.173650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:13.215908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e22c02-91cb-4f74-a9c1-ad6e946c2986 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:13.221879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:13.222543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:13.223230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:13.223557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:13.225400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:13.287764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:13.289428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337497e-1f06-4be2-bcad-9dd92277ae09 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:13.299236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1121/2245] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:14.868224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:14.868693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:14.868826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:14.975257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:14.978933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:15.148949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b64529-a335-4dcd-9d1f-2cef7a74e920 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:15.148949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b64529-a335-4dcd-9d1f-2cef7a74e920 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:15.148949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b64529-a335-4dcd-9d1f-2cef7a74e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:15.176697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1125/2246] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:15.180641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-35b64529-a335-4dcd-9d1f-2cef7a74e920 None None] MFA Rules not processed for user `44e87fb6b6d44e62a73bbf490478b37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:15.182858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-35b64529-a335-4dcd-9d1f-2cef7a74e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:15.236297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:15.238125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:15.311671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1122/2247] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:15.435464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851f951-42c8-417c-87e5-c59d4c18ce06 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:15.446953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21bee5-5bac-4a31-a2f1-b87470887f8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:15.448429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21bee5-5bac-4a31-a2f1-b87470887f8e service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:15.449679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21bee5-5bac-4a31-a2f1-b87470887f8e service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:15.450244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21bee5-5bac-4a31-a2f1-b87470887f8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:15.468809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:15.474877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21bee5-5bac-4a31-a2f1-b87470887f8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:15.479237 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21bee5-5bac-4a31-a2f1-b87470887f8e service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:15.505049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1126/2248] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:52:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:52:17.512394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:17.517970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:17.518588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:17.559158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] MFA Rules not processed for user `4b019c1d50bc4772b8a60ca73ab78dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:17.560545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:17.579490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1123/2249] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:21.029513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee32e0-9baa-40cc-9d08-1161c3de191e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:21.030180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee32e0-9baa-40cc-9d08-1161c3de191e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:21.030180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee32e0-9baa-40cc-9d08-1161c3de191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:21.071968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-89ee32e0-9baa-40cc-9d08-1161c3de191e None None] MFA Rules not processed for user `65c434b614e34b1f9b79375d85720e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:21.075568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-89ee32e0-9baa-40cc-9d08-1161c3de191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:21.156189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:21.158542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:21.243026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1127/2250] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:21.293732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9732d32-0dba-4c03-a2a1-c5cf9016eb73 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:21.300421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:21.303084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:21.303084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:21.303084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] PATH_INFO: `/v3/users/41d8a0c9dc324b199ad2fd3c0dc8d2ba` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:21.304348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d8a0c9dc324b199ad2fd3c0dc8d2ba)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:21.306990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:21.329025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:21.688558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d8a0c9dc324b199ad2fd3c0dc8d2ba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:21.689969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d8a0c9dc324b199ad2fd3c0dc8d2ba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:21.691281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d8a0c9dc324b199ad2fd3c0dc8d2ba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:21.711384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d8a0c9dc324b199ad2fd3c0dc8d2ba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:21.721198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d8a0c9dc324b199ad2fd3c0dc8d2ba'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:21.895207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1124/2251] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:52:21 2022] DELETE /identity/v3/users/41d8a0c9dc324b199ad2fd3c0dc8d2ba => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:52:21.905522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:21.905522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:21.906255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:21.975921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:21.978145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:22.146523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1128/2252] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:22.191976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:22.194003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:22.329132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b01677-ffab-493e-951f-bcd1ab5a5273 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:22.342421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c63e7f-f503-4d09-9a56-bfba17be64ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:22.343651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c63e7f-f503-4d09-9a56-bfba17be64ae service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:22.344221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c63e7f-f503-4d09-9a56-bfba17be64ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:22.344752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c63e7f-f503-4d09-9a56-bfba17be64ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:22.371298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c63e7f-f503-4d09-9a56-bfba17be64ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:22.377539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c63e7f-f503-4d09-9a56-bfba17be64ae service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:22.398848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1125/2253] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:23.187026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:23.187026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:23.236224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23274cb9-0477-479c-8202-056e8c4a3277 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:23.250537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:23.253294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:23.253625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:23.253850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] PATH_INFO: `/v3/projects/d8a270d9d39c4b8d916906ec77e5dfdf` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:23.256904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a270d9d39c4b8d916906ec77e5dfdf)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:23.258898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:23.442179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a270d9d39c4b8d916906ec77e5dfdf', 'request_id': 'req-821eb737-1a24-4c86-aa41-8adeb730a630'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:23.458450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a270d9d39c4b8d916906ec77e5dfdf', 'request_id': 'req-821eb737-1a24-4c86-aa41-8adeb730a630'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:23.559626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a270d9d39c4b8d916906ec77e5dfdf', 'request_id': 'req-821eb737-1a24-4c86-aa41-8adeb730a630'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:23.614322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] The token cache is being invalidate because project d8a270d9d39c4b8d916906ec77e5dfdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:52:23.614322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:23.614322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:23.618077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1129/2254] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:52:23 2022] DELETE /identity/v3/projects/d8a270d9d39c4b8d916906ec77e5dfdf => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:32.391501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:33.594673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:38.981786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:38.982313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:38.982313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:39.002112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:39.003304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:39.071078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1126/2255] 10.222.0.58 () {58 vars in 967 bytes} [Fri May 27 03:52:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:39.084458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:39.085942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:39.162773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821eb737-1a24-4c86-aa41-8adeb730a630 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:39.168354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:39.169057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:39.169342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:39.169622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:39.180980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:39.182615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:39.192619 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1130/2256] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:52:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:52:42.501236 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:42.502667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:42.623242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa914e-316e-4cbf-ba55-b4c892ad34d8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:42.628991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:42.629674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:42.630008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:42.630339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:42.632127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:42.644646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:42.646284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:42.659047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1127/2257] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:42.673978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:42.676232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:42.743955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffbb59b-d89b-4893-91b2-9d79072e36af service swift] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:42.749378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:42.750244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:42.750523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:42.750802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] PATH_INFO: `/v3/users/76fb4368530e4954b3e02e98da278589` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:42.751966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76fb4368530e4954b3e02e98da278589)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:42.753981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:42.966521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76fb4368530e4954b3e02e98da278589'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:42.967230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76fb4368530e4954b3e02e98da278589'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:42.974257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76fb4368530e4954b3e02e98da278589'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:42.974904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76fb4368530e4954b3e02e98da278589'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.068848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-d75d5e22-d0cd-454d-aad7-e3392e867575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76fb4368530e4954b3e02e98da278589'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.092135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1131/2258] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:52:42 2022] DELETE /identity/v3/users/76fb4368530e4954b3e02e98da278589 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:43.389380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:43.403542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182ed0-492f-4b04-b812-6013f5cebab5 service placement] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:43.475351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:43.476667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:43.477077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:43.477431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] PATH_INFO: `/v3/projects/63afcf01a7bc4cb6a4d88d96977f7311` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:43.480192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63afcf01a7bc4cb6a4d88d96977f7311)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:43.483369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:43.501703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:43.671557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63afcf01a7bc4cb6a4d88d96977f7311', 'request_id': 'req-10e69c2a-99a2-4b34-b74a-24fd26c8af27'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.798540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63afcf01a7bc4cb6a4d88d96977f7311', 'request_id': 'req-10e69c2a-99a2-4b34-b74a-24fd26c8af27'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.813944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63afcf01a7bc4cb6a4d88d96977f7311', 'request_id': 'req-10e69c2a-99a2-4b34-b74a-24fd26c8af27'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.820942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] The token cache is being invalidate because project 63afcf01a7bc4cb6a4d88d96977f7311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:52:43.821549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.822237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-10e69c2a-99a2-4b34-b74a-24fd26c8af27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:52:43.823839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1128/2259] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:52:43 2022] DELETE /identity/v3/projects/63afcf01a7bc4cb6a4d88d96977f7311 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:43.829167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:43.829711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:43.829812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:43.852384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:43.852384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:43.856949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:43.931628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1132/2260] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:52:43.940544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:43.940893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:43.941206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:43.961477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:43.962899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:43.971772 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1129/2261] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:43.976688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:43.987119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf3597-eb3d-492c-9250-97015fb0aa6b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:43.997883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:43.998580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:43.998858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:43.999188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:43.999905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:52:44.000175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.001968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.021888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1133/2262] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:52:44.023333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:44.035846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7bef-14c1-4f76-b53a-574be9f7dfb7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:44.039315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:44.040000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.040285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.041026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.041393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.044065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.174103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1130/2263] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:52:44 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:52:44.177419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:44.188339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e81d9e-b76a-41c1-bef8-362f1bd406a3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:44.259311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:44.260116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.260604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.260950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.262147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.264281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.409467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1134/2264] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:52:44 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:52:44.413512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:44.427140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9015849-3b90-42ad-a962-768a8e8c06b6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:44.432708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:44.433697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.434229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.434911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.435910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.437803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.459328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1131/2265] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:52:44.463281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:44.474714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f09e0-341a-4b0b-ba5f-1f7eff283075 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:44.480307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3072e-f376-4d21-893f-3a336256eda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:44.481175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3072e-f376-4d21-893f-3a336256eda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.481462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3072e-f376-4d21-893f-3a336256eda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.481786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3072e-f376-4d21-893f-3a336256eda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.482626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3072e-f376-4d21-893f-3a336256eda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.484331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3072e-f376-4d21-893f-3a336256eda3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.495434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1135/2266] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:52:44.500310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:44.510935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808549-457b-423a-a852-a6f41f5682a2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:44.517160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:44.518054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.518436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.518596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] PATH_INFO: `/v3/projects/21d579d798644cfba0e40b03320a5274/users/b481b236e8f64d0ea02e8b1d8aaec966/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.535694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d579d798644cfba0e40b03320a5274, user_id=b481b236e8f64d0ea02e8b1d8aaec966, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.538621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.636391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1132/2267] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:52:44 2022] PUT /identity/v3/projects/21d579d798644cfba0e40b03320a5274/users/b481b236e8f64d0ea02e8b1d8aaec966/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:52:44.646909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6141-c5ec-4502-96b6-a862e306ae24 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.647227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6141-c5ec-4502-96b6-a862e306ae24 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.647514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d6141-c5ec-4502-96b6-a862e306ae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.678087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-972d6141-c5ec-4502-96b6-a862e306ae24 None None] MFA Rules not processed for user `b481b236e8f64d0ea02e8b1d8aaec966`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:44.679302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-972d6141-c5ec-4502-96b6-a862e306ae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:44.753974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1136/2268] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:44.758807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:44.760526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:44.826294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295fdaf-d758-46d0-8087-84e203280d11 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:44.830121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2f346-3e85-4551-9077-15c6228a80d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:44.830776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2f346-3e85-4551-9077-15c6228a80d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:44.831061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2f346-3e85-4551-9077-15c6228a80d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:44.831414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2f346-3e85-4551-9077-15c6228a80d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:44.841226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2f346-3e85-4551-9077-15c6228a80d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:44.842717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2f346-3e85-4551-9077-15c6228a80d9 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:44.852057 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1133/2269] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:52:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:47.794579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:47.795118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:47.795118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:47.801728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:47.813530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] MFA Rules not processed for user `b481b236e8f64d0ea02e8b1d8aaec966`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:47.814846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:47.821774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1137/2270] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:52:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:54.847801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:58.007716 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:18.309257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:18.310111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:18.310407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:18.333098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:18.350170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:18.424771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1134/2271] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:53:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:18.450126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:18.451117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:18.534190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ca907-0717-4761-a8ad-8bf8d18be73d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:18.539293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66822b8-9377-44b4-920d-4cd6866a197f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:18.539957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66822b8-9377-44b4-920d-4cd6866a197f service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:18.540245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66822b8-9377-44b4-920d-4cd6866a197f service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:18.540616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66822b8-9377-44b4-920d-4cd6866a197f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:18.551393 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66822b8-9377-44b4-920d-4cd6866a197f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:18.553248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66822b8-9377-44b4-920d-4cd6866a197f service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:18.562600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1138/2272] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:53:18.899648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:18.901475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:18.972904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241cb94-5478-4329-bc40-a67d8ce62aa9 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:18.977458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:18.978336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:18.978620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:18.978916 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] PATH_INFO: `/v3/users/0e53671165414fce9d81d31964db486f` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:18.980132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e53671165414fce9d81d31964db486f)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:18.982170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:19.209807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e53671165414fce9d81d31964db486f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:19.210503 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e53671165414fce9d81d31964db486f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:19.211127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e53671165414fce9d81d31964db486f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:19.224941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e53671165414fce9d81d31964db486f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:19.234011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e53671165414fce9d81d31964db486f'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:19.312573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1135/2273] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:53:18 2022] DELETE /identity/v3/users/0e53671165414fce9d81d31964db486f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:19.318408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f006639-1df8-4c00-b815-32c7bb4b13d5 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:19.318615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f006639-1df8-4c00-b815-32c7bb4b13d5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:19.318993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f006639-1df8-4c00-b815-32c7bb4b13d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:19.341534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0f006639-1df8-4c00-b815-32c7bb4b13d5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:19.341534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0f006639-1df8-4c00-b815-32c7bb4b13d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:19.404484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1139/2274] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:53:19.415703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:19.417579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:19.469818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:19.504951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03acdf5-b8fc-4baf-bb97-0657731dcd72 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:19.510070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:19.511129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:19.511570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:19.511947 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:19.535297 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:19.536730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:19.550000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1136/2275] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:19.899728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:19.906400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:19.906400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:19.919165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:19.920383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:19.981177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1140/2276] 10.222.0.58 () {58 vars in 967 bytes} [Fri May 27 03:53:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:19.994559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:19.996006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:20.052097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.054496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:20.070020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e714f0-c233-4cc3-acd9-070614a83aa6 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.072144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963757e-1b81-45bf-a9c3-7792717a511a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.075433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77bc64-14fa-4600-989d-a84e4fa29602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.076171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77bc64-14fa-4600-989d-a84e4fa29602 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.076459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77bc64-14fa-4600-989d-a84e4fa29602 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.076774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77bc64-14fa-4600-989d-a84e4fa29602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.078187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.079253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.079662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.080009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] PATH_INFO: `/v3/projects/a518e9aade704470ad83e19420cb0f5e` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.081695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a518e9aade704470ad83e19420cb0f5e)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.083808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.087229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77bc64-14fa-4600-989d-a84e4fa29602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.088987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77bc64-14fa-4600-989d-a84e4fa29602 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.098376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1137/2277] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:53:20.191781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:20.204321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a518e9aade704470ad83e19420cb0f5e', 'request_id': 'req-98125cc7-2ece-4fd2-81eb-16a3b4560829'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:20.210143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a518e9aade704470ad83e19420cb0f5e', 'request_id': 'req-98125cc7-2ece-4fd2-81eb-16a3b4560829'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:20.286370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a518e9aade704470ad83e19420cb0f5e', 'request_id': 'req-98125cc7-2ece-4fd2-81eb-16a3b4560829'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:20.299258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] The token cache is being invalidate because project a518e9aade704470ad83e19420cb0f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:53:20.299544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:20.300210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-98125cc7-2ece-4fd2-81eb-16a3b4560829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:20.301783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1141/2278] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:53:20 2022] DELETE /identity/v3/projects/a518e9aade704470ad83e19420cb0f5e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:53:20.306977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.307298 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.307583 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.329023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:20.329525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:20.397308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1138/2279] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:20.411786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.415308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.415308 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.431432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:20.432667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:20.441136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1142/2280] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:20.445327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.455489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928c571-bdcc-4d3c-96e3-10956e6b9317 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.460625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.461296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.461605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.461884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.462657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:53:20.462967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.464595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.476999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1139/2281] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:53:20.481280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.491386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375fd1b-a06c-450a-82d4-ef72b4ae2bc7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.503165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f33199-403e-4fe7-a732-c9f35a28e424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.503834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f33199-403e-4fe7-a732-c9f35a28e424 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.504113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f33199-403e-4fe7-a732-c9f35a28e424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.504509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f33199-403e-4fe7-a732-c9f35a28e424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.505238 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f33199-403e-4fe7-a732-c9f35a28e424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.507244 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f33199-403e-4fe7-a732-c9f35a28e424 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.656025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1143/2282] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:53:20 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:53:20.660737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.675021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5682e0d-69af-4350-8a55-2ef1d6f952d3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.720388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:20.737707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.738282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.738637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.739017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.739868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.742300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.769775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.770061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.770339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.789193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:20.791131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:20.854967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1144/2283] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:53:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:53:20.861926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1140/2284] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:53:20 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:53:20.863394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.863653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.865055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:20.876441 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca16ec-50b7-4b06-94eb-e8e268570129 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.882597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.883677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.883677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.883883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.884574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.886231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.897570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1141/2285] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:53:20.901461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.912894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7030d-5275-42be-a3fc-606c2da9d07c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.917913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.918576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.918915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.919201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.920008 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.922370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.934098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1142/2286] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:53:20.938468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:20.939886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71ce7b-5639-4dcf-83aa-f18eb8ff3cf4 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.945075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612819fa-e57e-4833-8840-41cee7b7c460 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.945734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612819fa-e57e-4833-8840-41cee7b7c460 service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.946019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612819fa-e57e-4833-8840-41cee7b7c460 service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.946297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612819fa-e57e-4833-8840-41cee7b7c460 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.949240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f7d3d-0126-42ec-97a8-792c5cf975a7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:20.958780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:20.958780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:20.959041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612819fa-e57e-4833-8840-41cee7b7c460 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.959137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:20.959137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] PATH_INFO: `/v3/projects/f93f4399b40d40f7a17797dd8f40a536/users/795253c0f365447490452beb5581d2dd/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:20.959334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612819fa-e57e-4833-8840-41cee7b7c460 service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:20.969341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1145/2287] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:53:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:53:20.978265 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93f4399b40d40f7a17797dd8f40a536, user_id=795253c0f365447490452beb5581d2dd, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:20.981246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:21.064969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1143/2288] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:53:20 2022] PUT /identity/v3/projects/f93f4399b40d40f7a17797dd8f40a536/users/795253c0f365447490452beb5581d2dd/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:53:21.075630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e3be-9489-4d8a-b2fd-58e64e3cedc1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:21.075922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e3be-9489-4d8a-b2fd-58e64e3cedc1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:21.076203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4e3be-9489-4d8a-b2fd-58e64e3cedc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:21.108122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6bf4e3be-9489-4d8a-b2fd-58e64e3cedc1 None None] MFA Rules not processed for user `795253c0f365447490452beb5581d2dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:21.109524 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6bf4e3be-9489-4d8a-b2fd-58e64e3cedc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:21.127552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:21.180067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1146/2289] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:21.192240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:21.194113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:21.256215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042f7ce-1b6c-4a27-93bb-7de374048b98 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:21.259933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:21.260609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:21.260899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:21.261179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:21.270280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:21.271714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:21.281270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1144/2290] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:21.631741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:21.631741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:21.631741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:21.665003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:21.666304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:21.739594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1147/2291] 10.222.0.58 () {58 vars in 967 bytes} [Fri May 27 03:53:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:53:21.753521 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:21.826460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729de6e-5790-48ae-aa0b-fa2ea6d2a683 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:21.832592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:21.833324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:21.833680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:21.834026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:21.847201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:21.848949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:21.857847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1145/2292] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:53:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:53:21.901790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:21.903259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:21.968905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a122-d146-4819-a1ed-34056fbda3bf None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:21.971190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:21.971858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:21.972144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:21.972419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:21.974316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:22.046350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:22.047850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270744c6-8333-45c2-b48f-16425283d00a service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:22.058604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1148/2293] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:24.736414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:24.738684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:24.783042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:24.856602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08089d99-a15f-4e5f-87cf-dfff7dcad6c8 service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:24.862282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:24.864867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:24.865160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:24.865437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] PATH_INFO: `/v3/users/b481b236e8f64d0ea02e8b1d8aaec966` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:24.870433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b481b236e8f64d0ea02e8b1d8aaec966)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:24.872439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:25.154198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b481b236e8f64d0ea02e8b1d8aaec966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:25.155219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b481b236e8f64d0ea02e8b1d8aaec966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:25.156598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b481b236e8f64d0ea02e8b1d8aaec966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:25.172023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b481b236e8f64d0ea02e8b1d8aaec966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:25.180227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b481b236e8f64d0ea02e8b1d8aaec966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:25.285852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1146/2294] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:53:24 2022] DELETE /identity/v3/users/b481b236e8f64d0ea02e8b1d8aaec966 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:25.290366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:25.291212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:25.291652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:25.311818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:25.313450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:25.318199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:25.397609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1149/2295] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:25.407867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:25.409912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:25.489903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fb15-473b-4b34-897b-e7a47d95358f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:25.493690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c7c2a-be48-4b7c-8d96-31fd97917582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:25.494363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c7c2a-be48-4b7c-8d96-31fd97917582 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:25.494648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c7c2a-be48-4b7c-8d96-31fd97917582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:25.494930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c7c2a-be48-4b7c-8d96-31fd97917582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:25.506175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c7c2a-be48-4b7c-8d96-31fd97917582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:25.508009 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c7c2a-be48-4b7c-8d96-31fd97917582 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:25.520160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1147/2296] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:25.791555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:25.793344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:25.810279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74744d4-ac3a-459d-8395-271cd67a74eb None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:25.815932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:25.823100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:25.823100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:25.823100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] PATH_INFO: `/v3/projects/21d579d798644cfba0e40b03320a5274` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:25.823100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d579d798644cfba0e40b03320a5274)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:25.823100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:25.961835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d579d798644cfba0e40b03320a5274', 'request_id': 'req-41f1efd2-a332-4e93-a2b2-71608e15bd65'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:25.968020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d579d798644cfba0e40b03320a5274', 'request_id': 'req-41f1efd2-a332-4e93-a2b2-71608e15bd65'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:26.046228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d579d798644cfba0e40b03320a5274', 'request_id': 'req-41f1efd2-a332-4e93-a2b2-71608e15bd65'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:26.058481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] The token cache is being invalidate because project 21d579d798644cfba0e40b03320a5274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:53:26.058714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:26.059358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-41f1efd2-a332-4e93-a2b2-71608e15bd65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:26.061053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1150/2297] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:53:25 2022] DELETE /identity/v3/projects/21d579d798644cfba0e40b03320a5274 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:26.066633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.067132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.067237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.085838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:26.087102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:26.126733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:26.147616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1148/2298] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:26.155212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.155500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.155882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.173941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:26.175145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:26.182953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1151/2299] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:26.188031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.198163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d694ae-74cc-4c19-9a00-e15ac2655ca2 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.203627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.204288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.204624 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.204906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.205636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:53:26.205946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.207897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.221140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1149/2300] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:53:26.224390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.233700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcae1e-cf5e-4e6d-ad4a-1422c7adf00e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.238607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.239258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.239544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.239828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.240551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.242492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.319630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1152/2301] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:53:26 2022] POST /identity/v3/projects => generated 397 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:53:26.323716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.334850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f95eee-b95a-46f4-85d8-b9aa9b647907 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.392083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.392784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.393076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.393357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.394063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.395992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.505457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1150/2302] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:53:26 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:53:26.516733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.527311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de77f38-588d-4292-b266-256b6483e2ce admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.532707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.533407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.533735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.534016 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.534744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.536369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.549515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1153/2303] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:53:26.553623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.564317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6000a-c7db-423f-9ff9-61061be464bd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.569099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bfb85-fe94-4685-a450-52414bf94979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.569801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bfb85-fe94-4685-a450-52414bf94979 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.570031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bfb85-fe94-4685-a450-52414bf94979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.570337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bfb85-fe94-4685-a450-52414bf94979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.571061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bfb85-fe94-4685-a450-52414bf94979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.572691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bfb85-fe94-4685-a450-52414bf94979 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.583155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1151/2304] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:53:26.587484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.597434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2ec84-2ad9-45d2-bc00-e720da617ce0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.601958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.602828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.603106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.603376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] PATH_INFO: `/v3/projects/2c7a25193bf246d4a66eb7d639ec7ac2/users/3973da8b61474bb7a31827ece4370a49/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.618535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7a25193bf246d4a66eb7d639ec7ac2, user_id=3973da8b61474bb7a31827ece4370a49, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.621280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.686686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1154/2305] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:53:26 2022] PUT /identity/v3/projects/2c7a25193bf246d4a66eb7d639ec7ac2/users/3973da8b61474bb7a31827ece4370a49/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:26.706689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.707028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.707318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.739678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] MFA Rules not processed for user `3973da8b61474bb7a31827ece4370a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:26.743098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:26.820223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1152/2306] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:26.832067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:26.833931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:26.867784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:26.893345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cd84e-c57b-4278-a2fe-f5d3e6e23616 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:26.897212 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15a7df-ab86-494e-85ed-4d034b4461de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:26.898093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15a7df-ab86-494e-85ed-4d034b4461de service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:26.898711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15a7df-ab86-494e-85ed-4d034b4461de service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:26.899116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15a7df-ab86-494e-85ed-4d034b4461de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:26.908366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15a7df-ab86-494e-85ed-4d034b4461de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:26.909879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15a7df-ab86-494e-85ed-4d034b4461de service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:26.918911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1155/2307] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:36.817012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:36.959927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=13 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:48.294542 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:48.356400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b82661-0bf4-4b18-b179-e8348e38662d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:48.361158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:48.361856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:48.362139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:48.362412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] PATH_INFO: `/v3/users/3973da8b61474bb7a31827ece4370a49` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:48.363739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3973da8b61474bb7a31827ece4370a49)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:48.371722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:48.546369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3973da8b61474bb7a31827ece4370a49'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:48.546942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3973da8b61474bb7a31827ece4370a49'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:48.547548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3973da8b61474bb7a31827ece4370a49'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:48.560453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3973da8b61474bb7a31827ece4370a49'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:48.567638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3973da8b61474bb7a31827ece4370a49'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:48.654470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1153/2308] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:53:48 2022] DELETE /identity/v3/users/3973da8b61474bb7a31827ece4370a49 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:48.658982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:48.659463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:48.659561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:48.678708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:48.679909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:48.741501 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1156/2309] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:48.751518 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:48.753042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:48.816653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215ef50-4d85-4f7e-b2d6-f6b60b8574a6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:48.820808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596acb0-c5ca-435e-87a3-b5011558a0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:48.821480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596acb0-c5ca-435e-87a3-b5011558a0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:48.821753 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596acb0-c5ca-435e-87a3-b5011558a0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:48.822085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596acb0-c5ca-435e-87a3-b5011558a0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:48.832307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596acb0-c5ca-435e-87a3-b5011558a0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:48.834332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596acb0-c5ca-435e-87a3-b5011558a0f7 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:48.844708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1154/2310] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:49.356843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:49.358338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:49.374275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69388534-edb8-4632-a9ce-36a970e4b530 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:49.380396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:49.382521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:49.382882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:49.383129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] PATH_INFO: `/v3/projects/2c7a25193bf246d4a66eb7d639ec7ac2` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:49.384585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7a25193bf246d4a66eb7d639ec7ac2)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:49.386604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:49.551604 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7a25193bf246d4a66eb7d639ec7ac2', 'request_id': 'req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:49.560119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7a25193bf246d4a66eb7d639ec7ac2', 'request_id': 'req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:49.634358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7a25193bf246d4a66eb7d639ec7ac2', 'request_id': 'req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:49.646631 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] The token cache is being invalidate because project 2c7a25193bf246d4a66eb7d639ec7ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:53:49.647022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:49.647688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ac2d2f69-e8a4-4613-b8df-ac5ff349283f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:49.649296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1157/2311] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:53:49 2022] DELETE /identity/v3/projects/2c7a25193bf246d4a66eb7d639ec7ac2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:49.654976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:49.655481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:49.655481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:49.674575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:49.675816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:49.695010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:49.744838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1155/2312] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:49.752481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:49.753007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:49.753101 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:49.772824 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:49.774126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:49.782523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1158/2313] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:49.788166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:49.804883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1b737-e4a9-41c0-9ae0-1b6e243f9bc6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:49.812130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:49.812838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:49.813131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:49.813415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:49.814158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:53:49.814471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:49.816219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:49.829213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1156/2314] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:53:49.867809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:49.882325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd8a87-dd99-41cc-8ca2-1f4267da17bb None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:49.891133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:49.891845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:49.892136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:49.892472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:49.893992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:49.898171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:50.084929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1159/2315] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:53:49 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:53:50.089129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:50.099765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291cc63-3ed4-4e21-aa29-a312e57816ce admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:50.155600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:50.156177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:50.156457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:50.156765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:50.157458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:50.159369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:50.240712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1157/2316] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:53:50 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:53:50.244378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:50.255081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49612694-7545-4df0-bfdb-435d15930a93 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:50.259874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:50.260555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:50.260839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:50.261108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:50.261820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:50.263458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:50.274988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1160/2317] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:53:50.278616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:50.289300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680c764-7fc6-480a-98a2-95132da3c50d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:50.293865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eb057-c023-4ebc-9f40-db2f3714f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:50.294520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eb057-c023-4ebc-9f40-db2f3714f20f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:50.294810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eb057-c023-4ebc-9f40-db2f3714f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:50.295091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eb057-c023-4ebc-9f40-db2f3714f20f admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:50.295832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eb057-c023-4ebc-9f40-db2f3714f20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:50.297484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eb057-c023-4ebc-9f40-db2f3714f20f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:50.307377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1158/2318] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:53:50.311406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:50.321566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657b34f-8a08-467f-9562-f2076e110a90 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:50.326334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:50.327203 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:50.327489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:50.327770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] PATH_INFO: `/v3/projects/23a9a973f147497f976d77d3a468087e/users/dda6484b1d0e4fa9a3154ec50e7132c8/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:50.342931 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a9a973f147497f976d77d3a468087e, user_id=dda6484b1d0e4fa9a3154ec50e7132c8, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:50.346447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:50.401404 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1161/2319] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:53:50 2022] PUT /identity/v3/projects/23a9a973f147497f976d77d3a468087e/users/dda6484b1d0e4fa9a3154ec50e7132c8/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:50.406351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:50.406573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:50.406861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:50.426476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:50.431219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:50.494670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1159/2320] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:50.501294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:50.503138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:50.533085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:50.578506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb6874-fe5a-45de-850f-cdc8bed9faca admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:50.582591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bb16b-0c6b-4598-a6c2-4d7677830c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:50.583268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb16b-0c6b-4598-a6c2-4d7677830c04 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:50.583596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb16b-0c6b-4598-a6c2-4d7677830c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:50.584445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb16b-0c6b-4598-a6c2-4d7677830c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:50.594380 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bb16b-0c6b-4598-a6c2-4d7677830c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:50.595836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bb16b-0c6b-4598-a6c2-4d7677830c04 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:50.605984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1162/2321] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:53.040233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:53.041804 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:53.054532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d774f4-a9d1-41cb-bb42-e6462a5bf359 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:53.059329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:53.060422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:53.061065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:53.061494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] PATH_INFO: `/v3/users/795253c0f365447490452beb5581d2dd` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:53.062934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=795253c0f365447490452beb5581d2dd)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:53.065132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:53.096826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:53.257048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795253c0f365447490452beb5581d2dd'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:53.257938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795253c0f365447490452beb5581d2dd'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:53.258467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795253c0f365447490452beb5581d2dd'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:53.271095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795253c0f365447490452beb5581d2dd'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:53.280170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795253c0f365447490452beb5581d2dd'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:53.356470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1160/2322] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:53:53 2022] DELETE /identity/v3/users/795253c0f365447490452beb5581d2dd => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:53.357883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:53.358355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:53.359666 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:53.378658 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:53.380062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:53.454283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1163/2323] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:53.470729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:53.473044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:53.556139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b0db3-e75e-4dea-8bc2-bfd523c2ca88 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:53.560145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19d315-32db-41a7-8e48-aa16cbb65818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:53.560936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19d315-32db-41a7-8e48-aa16cbb65818 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:53.561275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19d315-32db-41a7-8e48-aa16cbb65818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:53.561599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19d315-32db-41a7-8e48-aa16cbb65818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:53.574472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19d315-32db-41a7-8e48-aa16cbb65818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:53.577922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19d315-32db-41a7-8e48-aa16cbb65818 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:53.589235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1161/2324] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:53:54.261255 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:54.262906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:54.276851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4a383-57b9-417d-82ba-88dc76268c50 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:54.289518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:54.290442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:54.290725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:54.291546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] PATH_INFO: `/v3/projects/f93f4399b40d40f7a17797dd8f40a536` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:54.292529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93f4399b40d40f7a17797dd8f40a536)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:54.294643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:54.441857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93f4399b40d40f7a17797dd8f40a536', 'request_id': 'req-1f9a293d-82e1-40a6-9f88-114712eaad70'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:54.450591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93f4399b40d40f7a17797dd8f40a536', 'request_id': 'req-1f9a293d-82e1-40a6-9f88-114712eaad70'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:54.547320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93f4399b40d40f7a17797dd8f40a536', 'request_id': 'req-1f9a293d-82e1-40a6-9f88-114712eaad70'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:54.569776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] The token cache is being invalidate because project f93f4399b40d40f7a17797dd8f40a536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:53:54.570076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:54.570781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:53:54.572425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1164/2325] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:53:54 2022] DELETE /identity/v3/projects/f93f4399b40d40f7a17797dd8f40a536 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:56.804333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:56.804936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:56.804936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:56.834635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] MFA Rules not processed for user `dda6484b1d0e4fa9a3154ec50e7132c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:53:56.835882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:56.906912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1162/2326] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:53:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:53:56.911450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:56.914233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:53:56.966872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:56.987762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a293d-82e1-40a6-9f88-114712eaad70 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:56.988133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:56.988901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:56.989191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:56.989467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:56.991266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:56.993359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:57.078782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1165/2327] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:53:56 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:53:57.079924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:57.091015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a990d-715e-46b4-b600-55988f6aa67f None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:57.121479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:57.149077 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:57.149798 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:57.150116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:57.150400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:57.151108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:57.153094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:57.229550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1163/2328] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:53:57 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:53:57.236689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:57.244809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f396a-8643-4b50-8841-3fd743b08d84 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:57.249563 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:57.250239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:57.250523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:57.250798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:57.251529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:57.253291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:57.268561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1166/2329] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:53:57.269651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:57.280106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca16105-b1cd-4aa5-8a41-9558b8338df3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:57.286251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:57.287641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:57.292442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:57.292442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] PATH_INFO: `/v3/projects/194aef5c052249e19ad7a83c107c0d42/users/84a8df877f5e47098a717a08921607ca/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:57.304862 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194aef5c052249e19ad7a83c107c0d42, user_id=84a8df877f5e47098a717a08921607ca, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:57.307800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:57.383285 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1164/2330] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:53:57 2022] PUT /identity/v3/projects/194aef5c052249e19ad7a83c107c0d42/users/84a8df877f5e47098a717a08921607ca/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:53:57.387317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:57.398661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97307d-d996-47ee-a4f3-81271f2db890 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:57.477932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2982163-5020-4a81-8311-84c089bf380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:57.478561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2982163-5020-4a81-8311-84c089bf380d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:57.478853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2982163-5020-4a81-8311-84c089bf380d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:57.479126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2982163-5020-4a81-8311-84c089bf380d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:57.479855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2982163-5020-4a81-8311-84c089bf380d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:57.481509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2982163-5020-4a81-8311-84c089bf380d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:57.492889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1167/2331] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:53:57.497790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:53:57.511751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc5ee0-6f4b-4cd6-b2fd-80aa56a290f4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:53:57.516812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:53:57.517740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:53:57.518070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:53:57.518368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] PATH_INFO: `/v3/projects/194aef5c052249e19ad7a83c107c0d42/users/84a8df877f5e47098a717a08921607ca/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:53:57.520370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194aef5c052249e19ad7a83c107c0d42, user_id=84a8df877f5e47098a717a08921607ca, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:53:57.523270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:53:57.534796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1165/2332] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:53:57 2022] PUT /identity/v3/projects/194aef5c052249e19ad7a83c107c0d42/users/84a8df877f5e47098a717a08921607ca/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:54:03.687662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc86f8-1f42-4911-b3d5-62043f569610 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:03.688526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc86f8-1f42-4911-b3d5-62043f569610 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:03.688968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc86f8-1f42-4911-b3d5-62043f569610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:03.728906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-35bc86f8-1f42-4911-b3d5-62043f569610 None None] MFA Rules not processed for user `84a8df877f5e47098a717a08921607ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:03.730529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-35bc86f8-1f42-4911-b3d5-62043f569610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:03.832806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1168/2333] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:54:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:54:03.845989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:03.850317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:03.978374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0582-1c49-4332-933b-f93349b91fd2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:03.982392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:03.983145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:03.983437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:03.983724 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:04.058517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:04.060064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:04.070181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1166/2334] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:04.265002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d6cfb-f0c0-4085-81bf-60abd574058d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:04.265210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d6cfb-f0c0-4085-81bf-60abd574058d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:04.265486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d6cfb-f0c0-4085-81bf-60abd574058d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:04.285521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ed1d6cfb-f0c0-4085-81bf-60abd574058d None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:04.287323 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ed1d6cfb-f0c0-4085-81bf-60abd574058d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:04.335534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:04.355383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1169/2335] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:54:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:54:04.369483 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:04.371025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:04.394574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:04.443743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee05dc-c3b5-4ec0-83de-f896b70e7b3d service glance] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:04.448692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:04.450126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:04.450126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:04.450126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:04.459669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:04.460899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:04.470601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1167/2336] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:54:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:54:08.173963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7341c6-303f-4c22-bfde-9fae357dc033 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:08.174592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7341c6-303f-4c22-bfde-9fae357dc033 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:08.174592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7341c6-303f-4c22-bfde-9fae357dc033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:08.194231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bb7341c6-303f-4c22-bfde-9fae357dc033 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:08.195448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bb7341c6-303f-4c22-bfde-9fae357dc033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:08.206699 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1170/2337] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:54:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:54:08.222152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:08.235990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe081c98-7470-4bfa-bc52-19e42a2d703c service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:08.242854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:08.243517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:08.243813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:08.244130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:08.257055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:08.259132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:08.270901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1168/2338] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:54:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:54:18.105901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31796964-0f79-4542-8762-1401725d4e6d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:18.106539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31796964-0f79-4542-8762-1401725d4e6d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:18.106539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31796964-0f79-4542-8762-1401725d4e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:18.125410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-31796964-0f79-4542-8762-1401725d4e6d None None] MFA Rules not processed for user `dda6484b1d0e4fa9a3154ec50e7132c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:18.126634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-31796964-0f79-4542-8762-1401725d4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:18.134264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1171/2339] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:54:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:54:18.264988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=29 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:19.992626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:19.995198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:20.062095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaa15f-f5e3-42ae-8158-109bebebc412 service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:20.066074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:20.066711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:20.067199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:20.067612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:20.069917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:20.172080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:20.173723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186755a0-422e-4f40-8bfb-255bb631d694 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:20.196099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1169/2340] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:21.426228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31796964-0f79-4542-8762-1401725d4e6d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:21.427699 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-31796964-0f79-4542-8762-1401725d4e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:21.439718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31796964-0f79-4542-8762-1401725d4e6d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:21.447761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:21.448861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:21.449336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:21.449813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:21.456795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:21.471629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:21.473152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:21.487721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1172/2341] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:54:23.248477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:23.248477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:23.248477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:23.266774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:23.268055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:23.278011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1170/2342] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:54:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:54:23.290836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:23.304871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8862064-d818-4d34-8cbd-a58bee615c87 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:23.310365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:23.311024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:23.311499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:23.311785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:23.322856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:23.324954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:23.334525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1173/2343] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:54:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:54:33.262721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=57 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:33.333121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=24 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:39.503313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:39.519628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af9f75-2634-4256-99b2-6dbd9b88b956 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:39.525166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:39.525839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:39.528283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:39.528283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:39.528480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:39.595013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:39.596549 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:39.607326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1171/2344] 10.222.0.58 () {60 vars in 1396 bytes} [Fri May 27 03:54:39 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:39.734694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:39.749261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa72fb8-a28a-4e38-a1e6-114a5d3fde00 service swift] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:39.756288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:39.767723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:39.767723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:39.767723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:39.767723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:39.772776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:39.774476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:39.783155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1174/2345] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:39.784128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:39.794362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6749a81-0328-4608-9b5b-90b9eb0396b5 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:39.798351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca180be9-a873-4b5c-af4a-3165686dc4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:39.799012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180be9-a873-4b5c-af4a-3165686dc4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:39.799296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180be9-a873-4b5c-af4a-3165686dc4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:39.799588 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180be9-a873-4b5c-af4a-3165686dc4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:39.808268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca180be9-a873-4b5c-af4a-3165686dc4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:39.809857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca180be9-a873-4b5c-af4a-3165686dc4f4 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:39.819187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1172/2346] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:54:49.462652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:49.473978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09e47f-adc7-4945-9f85-2b27bbfc66f7 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:49.476894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:49.477635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:49.477922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:49.478197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:49.480610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:49.493503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:49.497189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b691d3d-75a0-466c-a30d-781422e2b6a4 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:49.511688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1175/2347] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:49.814885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=34 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:49.984010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8f962-274d-438c-a99e-d08580e4fdaa None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:49.984310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8f962-274d-438c-a99e-d08580e4fdaa None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:49.984622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8f962-274d-438c-a99e-d08580e4fdaa None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:49.986306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1173/2348] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:54:49.995078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7653d-d1d0-42ce-b8ee-4c8599eaeee3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:49.995078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7653d-d1d0-42ce-b8ee-4c8599eaeee3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:49.995078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7653d-d1d0-42ce-b8ee-4c8599eaeee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.035435 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0df7653d-d1d0-42ce-b8ee-4c8599eaeee3 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:50.036779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0df7653d-d1d0-42ce-b8ee-4c8599eaeee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:50.046841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1176/2349] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:54:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:54:50.059092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8f962-274d-438c-a99e-d08580e4fdaa None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:50.071347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8f962-274d-438c-a99e-d08580e4fdaa None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:50.074898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472cd60-b8c3-4bd4-a317-222c9acf9c45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:50.075548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472cd60-b8c3-4bd4-a317-222c9acf9c45 service glance] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.075833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472cd60-b8c3-4bd4-a317-222c9acf9c45 service glance] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.076120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472cd60-b8c3-4bd4-a317-222c9acf9c45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.087675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472cd60-b8c3-4bd4-a317-222c9acf9c45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:50.089195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472cd60-b8c3-4bd4-a317-222c9acf9c45 service glance] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:50.102281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1174/2350] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:50.314665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505b1f9-2d09-4c31-9e12-8f4524d2dd10 None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.315276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505b1f9-2d09-4c31-9e12-8f4524d2dd10 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.315876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505b1f9-2d09-4c31-9e12-8f4524d2dd10 None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.319693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1177/2351] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:54:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:54:50.325490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477996d7-8f03-4482-8a56-eaeb146c8dd6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.325883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477996d7-8f03-4482-8a56-eaeb146c8dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.326237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477996d7-8f03-4482-8a56-eaeb146c8dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.357876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-477996d7-8f03-4482-8a56-eaeb146c8dd6 None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:50.359095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-477996d7-8f03-4482-8a56-eaeb146c8dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:50.367369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1175/2352] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:54:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:54:50.379275 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505b1f9-2d09-4c31-9e12-8f4524d2dd10 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:50.393160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505b1f9-2d09-4c31-9e12-8f4524d2dd10 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:50.396903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab06ef3-aa93-42a4-a7fc-ea225d52bf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:50.397556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab06ef3-aa93-42a4-a7fc-ea225d52bf43 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.397840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab06ef3-aa93-42a4-a7fc-ea225d52bf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.398119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab06ef3-aa93-42a4-a7fc-ea225d52bf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.406042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab06ef3-aa93-42a4-a7fc-ea225d52bf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:50.407491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab06ef3-aa93-42a4-a7fc-ea225d52bf43 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:50.417064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1178/2353] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:50.515710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b66b5f-9479-43c1-a0db-3acee6ae9096 None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.516111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b66b5f-9479-43c1-a0db-3acee6ae9096 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.516300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b66b5f-9479-43c1-a0db-3acee6ae9096 None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.517791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1176/2354] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:54:50.525195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.525730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.526134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.560531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:54:50.561788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:50.572543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1179/2355] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:54:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:54:50.584290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b66b5f-9479-43c1-a0db-3acee6ae9096 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:50.595607 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b66b5f-9479-43c1-a0db-3acee6ae9096 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:50.600073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbe158-8e35-4a1e-b57c-55f23bb6ffb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:50.600522 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbe158-8e35-4a1e-b57c-55f23bb6ffb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.600981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbe158-8e35-4a1e-b57c-55f23bb6ffb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.601205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbe158-8e35-4a1e-b57c-55f23bb6ffb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.611866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbe158-8e35-4a1e-b57c-55f23bb6ffb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:50.613348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbe158-8e35-4a1e-b57c-55f23bb6ffb0 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:50.625725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1177/2356] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:50.694904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:50.696305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:50.767175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb0023-04b7-4a47-8c5e-4d80119d21f5 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:50.772214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:50.772895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.773176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.773451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.775348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:50.789148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:50.790899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:50.801711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1180/2357] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:55:00.620575 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=15 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:00.797369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=62 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:01.614838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9aff-2a52-4354-a023-a913a923ef6e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:01.615135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9aff-2a52-4354-a023-a913a923ef6e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:01.615427 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb9aff-2a52-4354-a023-a913a923ef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:01.634424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-ccfb9aff-2a52-4354-a023-a913a923ef6e None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:55:01.636177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ccfb9aff-2a52-4354-a023-a913a923ef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:01.643594 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1178/2358] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:55:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:55:01.655448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:01.667680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9bbcf-aac0-4d70-8ce6-9d0985c81ad8 service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:01.672792 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:01.673437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:01.673731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:01.675764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:01.683616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:01.685260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:01.693584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1181/2359] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:55:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:55:03.273932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:03.274600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:03.274600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:03.294175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] MFA Rules not processed for user `44e87fb6b6d44e62a73bbf490478b37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:55:03.295411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:03.356526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1179/2360] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:55:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:55:03.607935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:03.617777 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1b047-a1d7-4925-8cfb-88f535edd612 service swift] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:03.625573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:03.626242 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:03.626524 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:03.626797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:03.628664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:03.639914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:03.641428 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:03.654290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1182/2361] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:55:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:55:13.347660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=28 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:13.646547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=12 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:21.713026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:21.715229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:21.733303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3d33c-54b1-4614-97f7-3bc7f73a0e11 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:21.740613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:21.743888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:21.743888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:21.743888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:21.744381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:21.758433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:21.760314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:21.774253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1180/2362] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:55:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:55:29.028033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:29.044366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f61c1-55bc-4077-b29a-b39a2cf740b3 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:29.050770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:29.051454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:29.051739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:29.052013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:29.053981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:29.067948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:29.069791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:29.082735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1183/2363] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:55:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:55:30.887626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:30.902897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e34d5-eaba-488c-ae3e-8201d915a0b2 service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:30.902897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bc0ee-f699-4039-8e1e-98e28f38b55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:30.902897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bc0ee-f699-4039-8e1e-98e28f38b55c service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:30.902897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bc0ee-f699-4039-8e1e-98e28f38b55c service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:30.903423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bc0ee-f699-4039-8e1e-98e28f38b55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:30.913711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bc0ee-f699-4039-8e1e-98e28f38b55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:30.915305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bc0ee-f699-4039-8e1e-98e28f38b55c service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:30.926437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1181/2364] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:55:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:55:33.463971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:33.465439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:33.479745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3c4e7-4ddc-4986-9a88-b311fd7a17be service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:33.486823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:33.487467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:33.487750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:33.488025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:33.489983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:33.504911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:33.506508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:33.517657 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1184/2365] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:55:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:55:36.253041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:36.253524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:36.253605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:36.273181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:55:36.274411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:36.282807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1182/2366] 10.222.0.58 () {58 vars in 967 bytes} [Fri May 27 03:55:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:55:36.305810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:36.318339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16041-c976-41eb-bc75-d9b3cfb0c8e7 service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:36.323410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:36.336531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:36.336531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:36.336531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:36.336531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:36.336531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:36.345003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1185/2367] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:55:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:55:46.267733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=15 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:46.340897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=18 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:53.730239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:55:53.743871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c62b10-0c57-41c2-bd3c-452a139355ce None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:55:53.749456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:55:53.750105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:55:53.750394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:55:53.750676 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:55:53.752545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:55:53.814210 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:55:53.815813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b84b0-2391-4e22-a681-29380a598949 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:55:53.827243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1183/2368] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:55:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:56:03.822657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=28 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:56:14.713419 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:56:14.725066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c994f5e-1cb6-45ef-bd98-05a1d7303836 service swift] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:56:14.730731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:56:14.731385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:56:14.731670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:56:14.731946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:56:14.733799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:56:14.795642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:56:14.797516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:56:14.808861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1186/2369] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:56:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:56:24.802891 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=28 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:56:30.287554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:56:30.288113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:56:30.288202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:56:30.305396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] MFA Rules not processed for user `65c434b614e34b1f9b79375d85720e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:56:30.306631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:56:30.313892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1184/2370] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:56:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:56:39.252395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:56:39.254857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:56:39.273240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f668bb0-dbdd-4a5b-902e-b315e5aa040f service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:56:39.280551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:56:39.280551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:56:39.280551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:56:39.280551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:56:39.282334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:56:39.295167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:56:39.298620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:56:39.308312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1187/2371] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:56:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:56:39.396353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:56:39.398367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:56:39.488815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8134e-4208-4f37-b852-ba201600d55d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:56:39.493461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:56:39.495167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:56:39.495167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:56:39.495167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] PATH_INFO: `/v3/users/44e87fb6b6d44e62a73bbf490478b37e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:56:39.496250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e87fb6b6d44e62a73bbf490478b37e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:56:39.498235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:56:39.707317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e87fb6b6d44e62a73bbf490478b37e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:39.707977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e87fb6b6d44e62a73bbf490478b37e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:39.708690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e87fb6b6d44e62a73bbf490478b37e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:39.721275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e87fb6b6d44e62a73bbf490478b37e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:39.728294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-31bc18f9-55ae-4600-be64-9277a42362c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e87fb6b6d44e62a73bbf490478b37e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:39.851291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1185/2372] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:56:39 2022] DELETE /identity/v3/users/44e87fb6b6d44e62a73bbf490478b37e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:56:40.236865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:56:40.253565 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad4e83-2847-492b-95f5-9436011c9b2c service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:56:40.319625 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:56:40.320528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:56:40.320821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:56:40.321099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] PATH_INFO: `/v3/projects/aa040dcc21404116bd2e8b5c782df93c` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:56:40.322318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa040dcc21404116bd2e8b5c782df93c)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:56:40.324327 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:56:40.447140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa040dcc21404116bd2e8b5c782df93c', 'request_id': 'req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:40.456578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa040dcc21404116bd2e8b5c782df93c', 'request_id': 'req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:40.554725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa040dcc21404116bd2e8b5c782df93c', 'request_id': 'req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:40.567337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] The token cache is being invalidate because project aa040dcc21404116bd2e8b5c782df93c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:56:40.567626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:40.568286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:56:40.569920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1188/2373] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:56:40 2022] DELETE /identity/v3/projects/aa040dcc21404116bd2e8b5c782df93c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:56:49.847619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:56:50.564060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:57:10.967972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:57:10.968612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:57:10.968612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:57:10.986796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] MFA Rules not processed for user `3cfb66d6bc254e549d6d355c2bbb440a`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:57:10.992178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:57:11.048746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1186/2374] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:57:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:57:13.395516 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:57:13.397134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:57:13.478829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a62e36-ad5f-4707-bffd-a1ca2a4d904e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:57:13.484580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:57:13.485251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:57:13.485534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:57:13.485808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:57:13.487662 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:57:13.550526 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:57:13.552037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecde1-33ba-40be-afd4-e0db0014838a service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:57:13.561813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1189/2375] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:57:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:57:21.041665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=25 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:57:23.557668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=56 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:57:52.017825 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:57:52.020860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:57:52.117907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be90dbc-45b0-469e-a019-dbbccea1b6cd None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:57:52.123117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:57:52.123812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:57:52.124092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:57:52.124363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:57:52.127939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:57:52.141869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:57:52.144880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:57:52.158423 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1187/2376] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:57:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:57:56.590605 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:57:56.591217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:57:56.591217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:57:56.610131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] MFA Rules not processed for user `bca16fe930244e12bbd12a23323d583e`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:57:56.611744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:57:56.667522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1190/2377] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:57:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:58:02.151786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=32 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:06.660930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=25 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:08.390237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:08.392519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:08.456132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0d9f9-8e25-49cb-b9f2-4ec7d32b9c92 service placement] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:08.459843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:08.460489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:08.460817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:08.461100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:08.463090 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:08.530325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:08.531790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:08.541997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1188/2378] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:58:13.644949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:13.647578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:13.665762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e04246-e80a-46be-a874-45d50b7e3575 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:13.670746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:13.671638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:13.671968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:13.672751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] PATH_INFO: `/v3/users/65c434b614e34b1f9b79375d85720e72` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:13.674048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c434b614e34b1f9b79375d85720e72)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:13.676852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:13.908051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c434b614e34b1f9b79375d85720e72'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:13.909663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c434b614e34b1f9b79375d85720e72'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:13.916899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c434b614e34b1f9b79375d85720e72'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:13.917549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c434b614e34b1f9b79375d85720e72'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.013059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a139602f-9ec8-458c-9abb-d53269043b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c434b614e34b1f9b79375d85720e72'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.031146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1191/2379] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:58:13 2022] DELETE /identity/v3/users/65c434b614e34b1f9b79375d85720e72 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:14.290869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:14.303776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c129758-5ca2-47a3-9c27-5bcd91020d76 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:14.363783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:14.364682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:14.364965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:14.365239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] PATH_INFO: `/v3/projects/df0a2a8dca064a3ea1c20b7a32842000` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:14.366418 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0a2a8dca064a3ea1c20b7a32842000)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:14.368451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:14.546177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0a2a8dca064a3ea1c20b7a32842000', 'request_id': 'req-4e883e95-d51c-47c2-bdbe-2a8c86934465'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.549089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:14.681445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0a2a8dca064a3ea1c20b7a32842000', 'request_id': 'req-4e883e95-d51c-47c2-bdbe-2a8c86934465'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.688425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0a2a8dca064a3ea1c20b7a32842000', 'request_id': 'req-4e883e95-d51c-47c2-bdbe-2a8c86934465'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.700165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] The token cache is being invalidate because project df0a2a8dca064a3ea1c20b7a32842000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:58:14.700457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.701158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4e883e95-d51c-47c2-bdbe-2a8c86934465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:14.702824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1189/2380] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:58:14 2022] DELETE /identity/v3/projects/df0a2a8dca064a3ea1c20b7a32842000 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:14.714080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:14.714614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:14.714734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:14.733739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:58:14.734954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:14.793835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1192/2381] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:58:14.802224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:14.802471 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:14.802746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:14.821029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:58:14.822243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:14.830809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1190/2382] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:58:14.834998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:14.844761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b7a2c-51cb-43cd-999b-9d66a55f0b87 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:14.849893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:14.850543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:14.850850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:14.851125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:14.851845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:58:14.852173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:14.853829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:14.865396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1193/2383] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:58:14.870803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:14.881854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de161e-50d0-4984-8ba9-6dd550a75365 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:14.886507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:14.887165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:14.887454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:14.887733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:14.888429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:14.890381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.009691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1191/2384] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:58:14 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:58:15.013984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.024749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335db16f-1300-43ab-89d3-5023625bbc69 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.085629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.086220 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.086504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.086783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.087472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.089453 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.096093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:15.211071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1194/2385] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:58:15 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:58:15.215433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.226413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1490b-89d9-4c92-9c20-8ddd7dc59a3d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.231270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.236595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.236595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.236595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.236595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.236595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.247093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1192/2386] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:58:15.252571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.262896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424af36-a070-499c-a985-6258d0d4bf69 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.268765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9e0f2-ec6d-4d3f-a302-6b3665c88add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.270973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9e0f2-ec6d-4d3f-a302-6b3665c88add admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.270973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9e0f2-ec6d-4d3f-a302-6b3665c88add admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.270973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9e0f2-ec6d-4d3f-a302-6b3665c88add admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.270973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9e0f2-ec6d-4d3f-a302-6b3665c88add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.272371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9e0f2-ec6d-4d3f-a302-6b3665c88add admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.283069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1195/2387] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:58:15.288066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.299883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac7ef4-6438-4494-a872-1c490d5c352d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.304927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.305856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.306114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.306390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] PATH_INFO: `/v3/projects/125776d86f9d4beaa01bc79ae6a341e7/users/1a4d99ad11114a05a7a248046df5d92a/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.323789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=125776d86f9d4beaa01bc79ae6a341e7, user_id=1a4d99ad11114a05a7a248046df5d92a, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.326400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.430789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1193/2388] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:58:15 2022] PUT /identity/v3/projects/125776d86f9d4beaa01bc79ae6a341e7/users/1a4d99ad11114a05a7a248046df5d92a/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:15.441529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.441821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.442103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.468403 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] MFA Rules not processed for user `1a4d99ad11114a05a7a248046df5d92a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:58:15.469653 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:15.536732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1196/2389] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:58:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:58:15.541415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.615574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0daea-4535-42f8-a5da-553f1edcd8c9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.620556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.621225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.621519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.621802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.622515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.624433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.706614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1194/2390] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:58:15 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:58:15.711267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.723840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6378c-e2f6-41d6-a111-9009999ec107 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.781478 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.782141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.782423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.783261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.783883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.785843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.874520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1197/2391] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:58:15 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:58:15.876115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.890114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357d116-d027-4519-85cc-c663ac7947bd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.896033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.896761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.897091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.897369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.898099 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.899716 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:15.911211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1195/2392] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:58:15.915506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:15.926161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a805e-77bd-495a-8782-a3c349b31d33 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:15.930845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:15.931736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:15.932014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:15.932285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] PATH_INFO: `/v3/projects/9c9f900f7cea489fa927d7e25c607da8/users/fcce81c28f51478fa69a6e863f17576e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:15.947106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9f900f7cea489fa927d7e25c607da8, user_id=fcce81c28f51478fa69a6e863f17576e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:15.950059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:16.019521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1198/2393] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:58:15 2022] PUT /identity/v3/projects/9c9f900f7cea489fa927d7e25c607da8/users/fcce81c28f51478fa69a6e863f17576e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:16.023500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:16.033507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd848c3-23d7-4839-bf50-4ae8392d025f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:16.074778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:16.091782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc8b3a-4d56-4c62-af47-2a3813f253e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:16.092452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc8b3a-4d56-4c62-af47-2a3813f253e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:16.092774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc8b3a-4d56-4c62-af47-2a3813f253e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:16.093059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc8b3a-4d56-4c62-af47-2a3813f253e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:16.093788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc8b3a-4d56-4c62-af47-2a3813f253e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:16.095425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc8b3a-4d56-4c62-af47-2a3813f253e2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:16.106650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1196/2394] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:58:16.111112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:16.121775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af7131-1c4c-4dba-94d9-432c470312cb admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:16.126568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:16.127444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:16.127729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:16.128060 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] PATH_INFO: `/v3/projects/9c9f900f7cea489fa927d7e25c607da8/users/fcce81c28f51478fa69a6e863f17576e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:16.130322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9f900f7cea489fa927d7e25c607da8, user_id=fcce81c28f51478fa69a6e863f17576e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:16.133153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:16.143986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1199/2395] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:58:16 2022] PUT /identity/v3/projects/9c9f900f7cea489fa927d7e25c607da8/users/fcce81c28f51478fa69a6e863f17576e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:16.155100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:16.155505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:16.155856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:16.185108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] MFA Rules not processed for user `fcce81c28f51478fa69a6e863f17576e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:58:16.186358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:16.256760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1197/2396] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:58:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:58:16.268112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:16.270447 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:16.293726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:16.332021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62263e36-77a2-4631-a7c6-27678027a9f7 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:16.336057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13381fdc-bb1a-4a4d-a369-7c5f147aff18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:16.336749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13381fdc-bb1a-4a4d-a369-7c5f147aff18 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:16.337034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13381fdc-bb1a-4a4d-a369-7c5f147aff18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:16.337306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13381fdc-bb1a-4a4d-a369-7c5f147aff18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:16.347151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13381fdc-bb1a-4a4d-a369-7c5f147aff18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:16.348863 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13381fdc-bb1a-4a4d-a369-7c5f147aff18 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:16.358925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1200/2397] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:58:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:58:16.546021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:16.557480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9074-4126-4483-a86b-bde99d0c0327 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:16.561278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:16.561937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:16.562226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:16.562509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:16.622351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:16.623803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:16.633535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1198/2398] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:58:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:58:26.354905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=17 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:26.630156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=72 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:33.514715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:33.515204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:33.515306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:33.533955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:58:33.535194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:33.599183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1201/2399] 10.222.0.58 () {58 vars in 967 bytes} [Fri May 27 03:58:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:58:33.746874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:33.748532 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:33.823655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e54aa1-d01b-43c9-b163-4d94d3ed4ec3 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:33.828749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:33.829397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:33.829681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:33.829958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:33.838826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:33.840410 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:33.848380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1199/2400] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:58:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:58:42.567409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:42.569111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:42.640915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5985eaa-6cdf-4417-b78b-ba211661d36d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:42.645303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:42.646693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:42.646967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:42.647241 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] PATH_INFO: `/v3/users/dda6484b1d0e4fa9a3154ec50e7132c8` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:42.648451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dda6484b1d0e4fa9a3154ec50e7132c8)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:42.650396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:42.854217 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dda6484b1d0e4fa9a3154ec50e7132c8'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:42.854781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dda6484b1d0e4fa9a3154ec50e7132c8'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:42.861620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dda6484b1d0e4fa9a3154ec50e7132c8'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:42.862245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dda6484b1d0e4fa9a3154ec50e7132c8'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:42.920344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dda6484b1d0e4fa9a3154ec50e7132c8'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:42.939445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1202/2401] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:58:42 2022] DELETE /identity/v3/users/dda6484b1d0e4fa9a3154ec50e7132c8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:43.355320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:43.367456 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800c302-1a3e-4c75-845e-4e03aa5ef640 service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:43.428177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:43.429131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:43.429358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:43.429640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] PATH_INFO: `/v3/projects/23a9a973f147497f976d77d3a468087e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:43.430840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a9a973f147497f976d77d3a468087e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:43.432824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:43.544232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a9a973f147497f976d77d3a468087e', 'request_id': 'req-ff177cfc-64e3-462a-bd6c-43e0a70741f8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.632398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a9a973f147497f976d77d3a468087e', 'request_id': 'req-ff177cfc-64e3-462a-bd6c-43e0a70741f8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.639195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a9a973f147497f976d77d3a468087e', 'request_id': 'req-ff177cfc-64e3-462a-bd6c-43e0a70741f8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.650208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] The token cache is being invalidate because project 23a9a973f147497f976d77d3a468087e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:58:43.650501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.651146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.652791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1200/2402] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:58:43 2022] DELETE /identity/v3/projects/23a9a973f147497f976d77d3a468087e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:58:43.656720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:43.658153 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:43.671704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5040b26-3160-43a0-afd3-63cb4392418c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:43.693639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:43.730354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:43.731229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:43.731521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:43.731797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] PATH_INFO: `/v3/users/84a8df877f5e47098a717a08921607ca` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:43.733015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a8df877f5e47098a717a08921607ca)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:43.735088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:43.949142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a8df877f5e47098a717a08921607ca'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.949723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a8df877f5e47098a717a08921607ca'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.957041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a8df877f5e47098a717a08921607ca'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:43.957715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a8df877f5e47098a717a08921607ca'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.021251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a8df877f5e47098a717a08921607ca'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.040180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1203/2403] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:58:43 2022] DELETE /identity/v3/users/84a8df877f5e47098a717a08921607ca => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:44.287959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:58:44.289463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:58:44.302517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177cfc-64e3-462a-bd6c-43e0a70741f8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:58:44.320626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:44.361333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:58:44.362201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:58:44.362495 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:58:44.362765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] PATH_INFO: `/v3/projects/194aef5c052249e19ad7a83c107c0d42` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:58:44.363996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=194aef5c052249e19ad7a83c107c0d42)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:58:44.366538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:58:44.486986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194aef5c052249e19ad7a83c107c0d42', 'request_id': 'req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.566190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194aef5c052249e19ad7a83c107c0d42', 'request_id': 'req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.574038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194aef5c052249e19ad7a83c107c0d42', 'request_id': 'req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.585166 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] The token cache is being invalidate because project 194aef5c052249e19ad7a83c107c0d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:58:44.585452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.586082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e8829be0-2ffb-4ade-b715-6b3d9121fc5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:58:44.587644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1201/2404] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:58:44 2022] DELETE /identity/v3/projects/194aef5c052249e19ad7a83c107c0d42 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:58:54.035729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:58:54.581395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:11.359420 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:11.360899 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:11.374663 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fa407-f028-4e3a-90e7-54cfc8d355a5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:11.434243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:11.435056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:11.435344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:11.435624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] PATH_INFO: `/v3/users/1a4d99ad11114a05a7a248046df5d92a` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:11.436857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4d99ad11114a05a7a248046df5d92a)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:11.438794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:11.618438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4d99ad11114a05a7a248046df5d92a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:11.619143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4d99ad11114a05a7a248046df5d92a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:11.627156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4d99ad11114a05a7a248046df5d92a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:11.627838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4d99ad11114a05a7a248046df5d92a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:11.719321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4d99ad11114a05a7a248046df5d92a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:11.740735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1204/2405] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:11 2022] DELETE /identity/v3/users/1a4d99ad11114a05a7a248046df5d92a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:11.745584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:11.746089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:11.746185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:11.763756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:11.765040 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:11.836330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1202/2406] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:11.846720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:11.848168 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:11.915595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dcadf6-0f20-4765-97e8-d1f3f8d1e74b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:11.920112 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:11.920875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:11.921156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:11.921427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:11.931001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:11.932456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:11.942406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1205/2407] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:12.410618 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:12.412085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:12.425653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1f00c-368c-4ede-9071-56b17db5c01e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:12.430424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:12.431300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:12.431593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:12.432362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] PATH_INFO: `/v3/projects/125776d86f9d4beaa01bc79ae6a341e7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:12.434552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=125776d86f9d4beaa01bc79ae6a341e7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:12.436846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:12.582199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125776d86f9d4beaa01bc79ae6a341e7', 'request_id': 'req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:12.667482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125776d86f9d4beaa01bc79ae6a341e7', 'request_id': 'req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:12.675252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125776d86f9d4beaa01bc79ae6a341e7', 'request_id': 'req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:12.685941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] The token cache is being invalidate because project 125776d86f9d4beaa01bc79ae6a341e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:12.686231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:12.686888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:12.688464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1203/2408] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:12 2022] DELETE /identity/v3/projects/125776d86f9d4beaa01bc79ae6a341e7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:12.692570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:12.703297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05783b-f006-4705-9b05-cfa97b3995f3 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:12.721573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:12.758387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:12.759254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:12.759531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:12.759802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] PATH_INFO: `/v3/users/fcce81c28f51478fa69a6e863f17576e` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:12.761002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcce81c28f51478fa69a6e863f17576e)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:12.763567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:13.013304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcce81c28f51478fa69a6e863f17576e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.014043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcce81c28f51478fa69a6e863f17576e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.023440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcce81c28f51478fa69a6e863f17576e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.024234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcce81c28f51478fa69a6e863f17576e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.090794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcce81c28f51478fa69a6e863f17576e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.108622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1206/2409] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:12 2022] DELETE /identity/v3/users/fcce81c28f51478fa69a6e863f17576e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:13.279817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:13.283225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:13.340786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9da7-ec30-4fa3-b793-cf284a7a71d9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:13.385496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:13.400022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:13.400709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:13.401199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:13.401316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:13.403190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:13.468564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:13.470045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fa4a9-4425-4a0a-a192-5a9ce8ecef3e service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:13.480140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1204/2410] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:13.634374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:13.635822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:13.648872 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fcde4-5b0a-4a1e-b049-e9c759b2b9b1 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:13.654202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:13.655077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:13.655497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:13.655773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] PATH_INFO: `/v3/projects/9c9f900f7cea489fa927d7e25c607da8` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:13.657032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9f900f7cea489fa927d7e25c607da8)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:13.658979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:13.782598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9f900f7cea489fa927d7e25c607da8', 'request_id': 'req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.788026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9f900f7cea489fa927d7e25c607da8', 'request_id': 'req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.884770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9f900f7cea489fa927d7e25c607da8', 'request_id': 'req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.899592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] The token cache is being invalidate because project 9c9f900f7cea489fa927d7e25c607da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:13.899592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.899592 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0a608dc7-0e56-4e04-b1be-f94b3dbd51c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:13.900977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1207/2411] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:13 2022] DELETE /identity/v3/projects/9c9f900f7cea489fa927d7e25c607da8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:13.906814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:13.907116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:13.907401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:13.929992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:13.938630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:14.004718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1205/2412] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:14.014427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.014744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.015002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.033249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:14.034464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:14.042740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1208/2413] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:14.047568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.057850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d9afe-5b74-41d4-ac7c-1912dc768c77 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.063318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.064002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.064284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.064605 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.065387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:59:14.065672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.067307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.083425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1206/2414] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:59:14.084540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.094103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e92c87-82d0-418d-9f09-d12e08ec88c3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.098815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.099477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.099772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.100056 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.100908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.102842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.117083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:14.357186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1209/2415] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:14 2022] POST /identity/v3/projects => generated 363 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:14.358356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.368357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55934b5-ee5e-4ad9-a606-7213f281a4d8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.431161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.431907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.432196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.432475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.434319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.435487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.506470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1207/2416] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:14 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:14.510708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.522058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deefea0-af1a-4de9-bf4e-d4c25c3bb385 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.535507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.535507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.535507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.535507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.537410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.537410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.548550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1210/2417] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:14.550000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.560910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ee40-bb05-42be-be3c-5e42d11d455b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.566003 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58da0c8-3e57-4432-9b9d-f3a2047fb7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.566887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da0c8-3e57-4432-9b9d-f3a2047fb7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.567245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da0c8-3e57-4432-9b9d-f3a2047fb7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.567595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58da0c8-3e57-4432-9b9d-f3a2047fb7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.568400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58da0c8-3e57-4432-9b9d-f3a2047fb7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.570124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58da0c8-3e57-4432-9b9d-f3a2047fb7b1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.571974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:14.585162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1208/2418] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:14.586815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.603426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79e6e-e369-461e-b8d8-e791d391b760 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.610654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.612586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.614532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.614822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] PATH_INFO: `/v3/projects/be328d4cbde74bd19a0582286e989249/users/f96736acb7c94cea96d603c4a10cd966/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.632710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be328d4cbde74bd19a0582286e989249, user_id=f96736acb7c94cea96d603c4a10cd966, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.635552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.700231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1211/2419] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:14 2022] PUT /identity/v3/projects/be328d4cbde74bd19a0582286e989249/users/f96736acb7c94cea96d603c4a10cd966/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:14.711215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.711557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.711792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.742634 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] MFA Rules not processed for user `f96736acb7c94cea96d603c4a10cd966`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:14.743917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:14.811184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1209/2420] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:14.816083 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:14.880434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a20fe-13bf-4246-a54a-11ad8c7a7b8b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:14.885188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:14.885860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:14.886146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:14.886425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:14.887223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:14.889334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:14.987116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1212/2421] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:14 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:14.990727 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.003355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329dcf2-ec70-4eac-b50b-b3e3634aeb39 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.075803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.076647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.077038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.077414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.078635 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:15.080537 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:15.189022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1210/2422] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:14 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:15.197660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.210954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81d935-d91f-4f74-b035-722d529c9902 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.217841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.218812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.219048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.219438 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.220424 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:15.225979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:15.240169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1213/2423] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:15.241477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.253854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f697a1-e92f-4022-b4fc-2b9ef6515046 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.259332 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.260381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.260799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.261180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] PATH_INFO: `/v3/projects/d758205caad64fda87645ddbc8d08940/users/59298dce5a144610a41153786c50f774/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.282734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d758205caad64fda87645ddbc8d08940, user_id=59298dce5a144610a41153786c50f774, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:15.285742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:15.360688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1211/2424] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:15 2022] PUT /identity/v3/projects/d758205caad64fda87645ddbc8d08940/users/59298dce5a144610a41153786c50f774/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:15.364768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.376188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf14235-a20b-4458-b4fc-395a19e201be admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.437702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25789c5d-0a62-442c-890d-580eacbbb605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.438956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25789c5d-0a62-442c-890d-580eacbbb605 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.438956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25789c5d-0a62-442c-890d-580eacbbb605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.438956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25789c5d-0a62-442c-890d-580eacbbb605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.439635 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25789c5d-0a62-442c-890d-580eacbbb605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:15.441306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25789c5d-0a62-442c-890d-580eacbbb605 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:15.452552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1214/2425] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:15.456925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.467381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67599010-e27d-41de-9c9e-101f4fc65ae2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.472118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.473043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.473326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.473609 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] PATH_INFO: `/v3/projects/d758205caad64fda87645ddbc8d08940/users/59298dce5a144610a41153786c50f774/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.475732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] RBAC: Authorizing `identity:create_grant(project_id=d758205caad64fda87645ddbc8d08940, user_id=59298dce5a144610a41153786c50f774, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:15.478478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:15.489287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1212/2426] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:15 2022] PUT /identity/v3/projects/d758205caad64fda87645ddbc8d08940/users/59298dce5a144610a41153786c50f774/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:15.502513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.502808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.503090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.532703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] MFA Rules not processed for user `59298dce5a144610a41153786c50f774`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:15.534917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:15.558277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:15.613144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1215/2427] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:15.625299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.627589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:15.678776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:15.694694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3ad1c-7e73-4ee0-8fff-56dde03612af admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.698645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.699293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.699630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.699906 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:15.709202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:15.710661 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:15.721566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1213/2428] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:15.954429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:15.966387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b75f1-7042-4504-9120-d9412f882bf2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:15.970142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:15.970748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:15.971038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:15.971318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:16.057615 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:16.060079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:16.080655 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1216/2429] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:19.671188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:19.741045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb827c-b2d1-42be-8091-feedc98f9db1 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:19.745960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:19.747629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:19.747915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:19.748205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] PATH_INFO: `/v3/users/f96736acb7c94cea96d603c4a10cd966` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:19.749581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96736acb7c94cea96d603c4a10cd966)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:19.751581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:19.959406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:19.960996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:20.015631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96736acb7c94cea96d603c4a10cd966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:20.016372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96736acb7c94cea96d603c4a10cd966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:20.017023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96736acb7c94cea96d603c4a10cd966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:20.029197 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f34be-22e7-4081-a2b2-5b7eddb38012 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:20.033100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:20.033638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96736acb7c94cea96d603c4a10cd966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:20.033806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:20.034029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:20.034305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:20.036318 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:20.042750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96736acb7c94cea96d603c4a10cd966'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:20.131778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:20.135011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:20.137890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d537da-7acb-4c8e-bb0e-b236810b999a service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:20.152171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1217/2430] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:59:20.218729 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1214/2431] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:19 2022] DELETE /identity/v3/users/f96736acb7c94cea96d603c4a10cd966 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:20.220480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:20.220802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:20.221084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:20.243923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:20.245882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:20.329179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1218/2432] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:20.338631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:20.341260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:20.397545 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab8873-0771-422f-b714-43a6e50597ae admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:20.401239 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:20.401885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:20.402173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:20.403102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:20.412651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:20.414095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:20.430450 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1215/2433] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:20.921897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:20.923352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:20.938107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356b6f3-01a1-48e4-ae23-9bc252d7b1cb None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:20.944162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:20.945080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:20.945703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:20.945870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] PATH_INFO: `/v3/projects/be328d4cbde74bd19a0582286e989249` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:20.947104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be328d4cbde74bd19a0582286e989249)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:20.949065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:21.058569 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be328d4cbde74bd19a0582286e989249', 'request_id': 'req-89d29a2b-2bf3-47dc-88d1-d70a0956793c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.068566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be328d4cbde74bd19a0582286e989249', 'request_id': 'req-89d29a2b-2bf3-47dc-88d1-d70a0956793c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.134974 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be328d4cbde74bd19a0582286e989249', 'request_id': 'req-89d29a2b-2bf3-47dc-88d1-d70a0956793c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.150115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] The token cache is being invalidate because project be328d4cbde74bd19a0582286e989249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:21.150115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.151797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.153763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1219/2434] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:20 2022] DELETE /identity/v3/projects/be328d4cbde74bd19a0582286e989249 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:21.159029 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:21.165859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:21.173687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb7bca-7e58-493b-8ef6-2def80c2e1c9 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:21.241818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:21.242702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:21.242990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:21.243273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] PATH_INFO: `/v3/users/59298dce5a144610a41153786c50f774` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:21.244478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59298dce5a144610a41153786c50f774)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:21.247219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:21.411903 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59298dce5a144610a41153786c50f774'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.412669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59298dce5a144610a41153786c50f774'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.413501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59298dce5a144610a41153786c50f774'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.427467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59298dce5a144610a41153786c50f774'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.434872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59298dce5a144610a41153786c50f774'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.478340 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:21.480141 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:21.495956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d29a2b-2bf3-47dc-88d1-d70a0956793c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:21.501154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:21.502079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:21.502358 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:21.502634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] PATH_INFO: `/v3/users/4b019c1d50bc4772b8a60ca73ab78dd0` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:21.503900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b019c1d50bc4772b8a60ca73ab78dd0)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:21.506001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:21.552926 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1216/2435] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:21 2022] DELETE /identity/v3/users/59298dce5a144610a41153786c50f774 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:21.749182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b019c1d50bc4772b8a60ca73ab78dd0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.749832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b019c1d50bc4772b8a60ca73ab78dd0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.756888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b019c1d50bc4772b8a60ca73ab78dd0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.758316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b019c1d50bc4772b8a60ca73ab78dd0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.850330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b019c1d50bc4772b8a60ca73ab78dd0'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:21.872695 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:21.883719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1220/2436] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:21 2022] DELETE /identity/v3/users/4b019c1d50bc4772b8a60ca73ab78dd0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:22.046911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.048445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:22.130650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe43c7-a0a8-493d-afbf-d9a26fa31291 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:22.142661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.144249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:22.165807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:22.200850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:22.201720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.202042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.202377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] PATH_INFO: `/v3/projects/d758205caad64fda87645ddbc8d08940` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.203590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d758205caad64fda87645ddbc8d08940)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:22.205616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:22.222810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0903a2-5757-4d10-b5b0-cc1d680b663b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:22.229270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:22.230219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.230626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.230942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] PATH_INFO: `/v3/projects/2f97b57909174abeab16dedc077b37e0` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.232805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f97b57909174abeab16dedc077b37e0)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:22.235755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:22.326512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd758205caad64fda87645ddbc8d08940', 'request_id': 'req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.359284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f97b57909174abeab16dedc077b37e0', 'request_id': 'req-7056eb44-03f0-48c3-91d0-327ce038864f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.364999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f97b57909174abeab16dedc077b37e0', 'request_id': 'req-7056eb44-03f0-48c3-91d0-327ce038864f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.408534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd758205caad64fda87645ddbc8d08940', 'request_id': 'req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.415226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd758205caad64fda87645ddbc8d08940', 'request_id': 'req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.426490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] The token cache is being invalidate because project d758205caad64fda87645ddbc8d08940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:22.426779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.427425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ae4a1b01-48db-4e14-aaa1-b0d8e4e71b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.429023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1217/2437] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:22 2022] DELETE /identity/v3/projects/d758205caad64fda87645ddbc8d08940 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:22.434405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0fa09-bed1-4ce0-9ab7-dc2879c10086 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.434707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0fa09-bed1-4ce0-9ab7-dc2879c10086 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.435143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0fa09-bed1-4ce0-9ab7-dc2879c10086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.454515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8ea0fa09-bed1-4ce0-9ab7-dc2879c10086 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:22.455777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8ea0fa09-bed1-4ce0-9ab7-dc2879c10086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:22.472802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f97b57909174abeab16dedc077b37e0', 'request_id': 'req-7056eb44-03f0-48c3-91d0-327ce038864f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.485412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] The token cache is being invalidate because project 2f97b57909174abeab16dedc077b37e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:22.485861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.486521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.488054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1221/2438] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:22 2022] DELETE /identity/v3/projects/2f97b57909174abeab16dedc077b37e0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:22.491878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.493283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:22.519425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1218/2439] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:22.529037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.529348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.529962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.555860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:22.557259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:22.567151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056eb44-03f0-48c3-91d0-327ce038864f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:22.571082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1219/2440] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:22.572802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:22.573507 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.573903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.574391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.582630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] PATH_INFO: `/v3/users/ecd46537ac5246b191b605f9cf141710` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.582630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd46537ac5246b191b605f9cf141710)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:22.582630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:22.583326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde2bc6-2731-4840-be48-d6a5cb48b3c3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:22.588755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:22.589422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.589980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.590411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.591364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:59:22.591673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:22.593328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:22.604730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:22.610907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1220/2441] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:59:22.616940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.627270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e29de-ac51-4510-a42d-b683acfa51e6 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:22.632252 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:22.632978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.633235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.633514 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.634271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:22.636186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:22.748896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1221/2442] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:22 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:22.753002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.772974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:22.815799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0c6d0-123a-4c23-be65-50f9d635cc33 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:22.817129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd46537ac5246b191b605f9cf141710'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.817883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd46537ac5246b191b605f9cf141710'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.820774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:22.821434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:22.821719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:22.821995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:22.822682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:22.824933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:22.826542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd46537ac5246b191b605f9cf141710'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.827218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd46537ac5246b191b605f9cf141710'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.905064 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd46537ac5246b191b605f9cf141710'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:22.923126 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1222/2443] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:22 2022] DELETE /identity/v3/users/ecd46537ac5246b191b605f9cf141710 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:22.937989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1222/2444] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:22 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:22.942246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:22.944180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.009715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be0a85-0a41-493a-94df-776b4dd7ff34 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.016282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.017008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.017290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.017568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.018301 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.019970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.031462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1223/2445] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:23.035841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.046553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf5d520-4159-4dc3-9a4b-ed37cad9cb74 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.051581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.052335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.052660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.052944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.053686 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.055351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.066973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1223/2446] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:23.071227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.082337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc27eb-3b3e-4ff0-bb31-e0c8d6431888 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.087232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae792ed-1bdc-409f-b2a2-371445753685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.088116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae792ed-1bdc-409f-b2a2-371445753685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.088441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae792ed-1bdc-409f-b2a2-371445753685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.088809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae792ed-1bdc-409f-b2a2-371445753685 admin admin] PATH_INFO: `/v3/projects/14c4f8d86eae4a3bbaec5b8e587f4d1b/users/6c5066b6b7614a278c83f495de399d28/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.106202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae792ed-1bdc-409f-b2a2-371445753685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c4f8d86eae4a3bbaec5b8e587f4d1b, user_id=6c5066b6b7614a278c83f495de399d28, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.113885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae792ed-1bdc-409f-b2a2-371445753685 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.206325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1224/2447] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:23 2022] PUT /identity/v3/projects/14c4f8d86eae4a3bbaec5b8e587f4d1b/users/6c5066b6b7614a278c83f495de399d28/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:23.210814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.216030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.216319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.216628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.218508 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.233012 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fa29b-1ac9-4b31-929e-23d96aba0ecf admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.251396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] MFA Rules not processed for user `6c5066b6b7614a278c83f495de399d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:23.252696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.291368 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.292240 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.292548 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.294602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] PATH_INFO: `/v3/projects/e4e0a97e86df4bd38461b02c6b2db08c` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.295913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e0a97e86df4bd38461b02c6b2db08c)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.297918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.318889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1225/2448] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:23.323654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.330110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:23.335632 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed91ecb-f5ed-4ca0-ad16-b5ef3d31624c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.341370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.342019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.342307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.342585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.343272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.345209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.411494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e0a97e86df4bd38461b02c6b2db08c', 'request_id': 'req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:23.478394 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1226/2449] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:23.482611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.507682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e0a97e86df4bd38461b02c6b2db08c', 'request_id': 'req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:23.517327 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e0a97e86df4bd38461b02c6b2db08c', 'request_id': 'req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:23.533457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] The token cache is being invalidate because project e4e0a97e86df4bd38461b02c6b2db08c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:23.533942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:23.534808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ca6eba1d-6bd9-4868-b007-ff5cfc7049f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:23.536682 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1224/2450] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:23 2022] DELETE /identity/v3/projects/e4e0a97e86df4bd38461b02c6b2db08c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:23.541459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8fb2-5c5a-4cbe-857f-9613599e8499 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.541831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8fb2-5c5a-4cbe-857f-9613599e8499 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.542180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8fb2-5c5a-4cbe-857f-9613599e8499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.551834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9337018-4823-477f-a847-5ba1bf92e200 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.556750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.557431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.557712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.557989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.559369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.561135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-589a8fb2-5c5a-4cbe-857f-9613599e8499 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:23.561532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.562338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-589a8fb2-5c5a-4cbe-857f-9613599e8499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.570788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1225/2451] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:23.579353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.579647 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.579927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.598292 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:23.599476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.607396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1226/2452] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:23.611527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.620748 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bdf77-c8d3-4da9-b6c0-84c2a15ff601 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.626274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.627024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.627354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.633100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.633100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:59:23.633100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.633100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.639818 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:23.643397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1227/2453] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:59:23.647207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.657911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fbb4d-1941-4c94-b01e-04d223c6bc26 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.662118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1227/2454] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:23.665406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.665803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.666526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.666819 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.667005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.667111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.667801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.672536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.680190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167725ec-8ba5-4106-90b3-1fe899e4f84f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.685414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.686119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.686406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.686686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.687417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.689093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.700250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1228/2455] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:23.704855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.715315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2053b6d-ed3f-4ac7-9042-a4387cbe725e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.720679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.721344 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.721633 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.721915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.722649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.724349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.740096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1229/2456] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:23.740928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1228/2457] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:23.742473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.744722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.752361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25baf9-bc1d-40e7-a951-62726d064712 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.813072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.813875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.814166 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.814372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90485afd-a58e-4642-8b4e-490e987536dd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:23.814487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] PATH_INFO: `/v3/projects/3ca9b62811be45c884c901d6c1b8e52f/users/9718af7dbe0e43bbad1a8e833484a643/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.818918 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:23.819581 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:23.819870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:23.820402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:23.821126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.823064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.829871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca9b62811be45c884c901d6c1b8e52f, user_id=9718af7dbe0e43bbad1a8e833484a643, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:23.832913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:23.923535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1229/2458] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:23 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:23.924564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.926247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:23.945618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1230/2459] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:23 2022] PUT /identity/v3/projects/3ca9b62811be45c884c901d6c1b8e52f/users/9718af7dbe0e43bbad1a8e833484a643/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:23.946531 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:23.987034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:23.987566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba041998-b62b-497a-ace6-0740e2cf963f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.020575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02c3a3-5273-404f-a857-be97719df5d9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.026027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4dd3b-f106-486f-9baf-4ea8494585e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.026793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4dd3b-f106-486f-9baf-4ea8494585e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.027091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4dd3b-f106-486f-9baf-4ea8494585e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.027918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4dd3b-f106-486f-9baf-4ea8494585e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.029078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4dd3b-f106-486f-9baf-4ea8494585e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.030763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4dd3b-f106-486f-9baf-4ea8494585e6 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.038304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.038952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.039233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.039505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.041332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1231/2460] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:24.042433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.042719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.043028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.044401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:24.076369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] MFA Rules not processed for user `9718af7dbe0e43bbad1a8e833484a643`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:24.077532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:24.091699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:24.124536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.125714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.138761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1230/2461] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:24.139666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.144979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1232/2462] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:24.149688 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.150060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ce73-3beb-46ef-8d4e-1123705adcd8 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.156339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.157243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.157643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.158027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.158833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.160570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.163896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf7b9-eb70-4772-ad96-c8c8d15b09f1 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.174556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1231/2463] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:24.174750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.174750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.174750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.174750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.175142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.176299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.177351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.188074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc5230-5f5a-454e-8fcf-a4b1a8643086 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.193448 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51233a4-63fe-4b7e-8c84-57792c0441e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.194683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51233a4-63fe-4b7e-8c84-57792c0441e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.195085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51233a4-63fe-4b7e-8c84-57792c0441e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.195963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51233a4-63fe-4b7e-8c84-57792c0441e9 admin admin] PATH_INFO: `/v3/projects/5e8a7647a47e4ac19972582346dbdcee/users/f1d4360de70b4beea3456aa3dc85d5ec/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.213072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51233a4-63fe-4b7e-8c84-57792c0441e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8a7647a47e4ac19972582346dbdcee, user_id=f1d4360de70b4beea3456aa3dc85d5ec, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.217076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51233a4-63fe-4b7e-8c84-57792c0441e9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.264697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1233/2464] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:24 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:24.265796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.305960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1232/2465] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:24 2022] PUT /identity/v3/projects/5e8a7647a47e4ac19972582346dbdcee/users/f1d4360de70b4beea3456aa3dc85d5ec/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:24.312263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.312774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.313220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.340067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:24.341843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:24.348214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eb2af-41ec-4022-920a-7c4e31e571c8 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.392352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:24.407805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1233/2466] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:24.408798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.409483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.409774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.410053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.410874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.412861 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.417434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.419030 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:24.476094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1175aa-a322-49da-bcb2-f0094285647d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.479774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.480763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.481047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.481333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.490346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.491781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.502119 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1234/2467] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:59:24.507968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1234/2468] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:24 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:24.514296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.524058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26837d47-f752-4999-a689-6ea3ff49fb2e service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.529130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.529828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.530100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.530388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.531127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.532832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.543675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1235/2469] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:24.547928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.558274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9c7d0-192e-4e87-814b-f689a7db128a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.563547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.564626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.564932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.565189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] PATH_INFO: `/v3/projects/20106c969af7479d8d18ced0b2150f42/users/500cf044a2a441c0b8f65de758cc939e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.581098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=20106c969af7479d8d18ced0b2150f42, user_id=500cf044a2a441c0b8f65de758cc939e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.583753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.662452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1235/2470] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:24 2022] PUT /identity/v3/projects/20106c969af7479d8d18ced0b2150f42/users/500cf044a2a441c0b8f65de758cc939e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:24.666466 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.678192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8971c8-f264-4d4a-b9bd-12a63d465f40 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.698651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:24.739780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1577a-c600-4261-8893-538c3cf87879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.740454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1577a-c600-4261-8893-538c3cf87879 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.740766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1577a-c600-4261-8893-538c3cf87879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.741067 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1577a-c600-4261-8893-538c3cf87879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.741975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1577a-c600-4261-8893-538c3cf87879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.743636 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1577a-c600-4261-8893-538c3cf87879 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.754808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1236/2471] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:24.759152 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.770711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e4b4-9911-4b98-92c1-aafe0f3594ee admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.775744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.776868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.777161 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.777623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] PATH_INFO: `/v3/projects/20106c969af7479d8d18ced0b2150f42/users/500cf044a2a441c0b8f65de758cc939e/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.779582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20106c969af7479d8d18ced0b2150f42, user_id=500cf044a2a441c0b8f65de758cc939e, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:24.782292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:24.793911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1236/2472] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:24 2022] PUT /identity/v3/projects/20106c969af7479d8d18ced0b2150f42/users/500cf044a2a441c0b8f65de758cc939e/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:24.804306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.804434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.804656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:24.832951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] MFA Rules not processed for user `500cf044a2a441c0b8f65de758cc939e`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:24.834181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:24.909889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1237/2473] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:24.921835 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:24.923556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:24.985536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0cd64-edef-4d5a-b9c0-f8abb061a5d0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:24.989234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13d8ed-a638-47ba-95d0-b990316a167b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:24.989883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d8ed-a638-47ba-95d0-b990316a167b service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:24.990168 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d8ed-a638-47ba-95d0-b990316a167b service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:24.990443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13d8ed-a638-47ba-95d0-b990316a167b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:25.002820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13d8ed-a638-47ba-95d0-b990316a167b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:25.004342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13d8ed-a638-47ba-95d0-b990316a167b service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:25.014544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1237/2474] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:59:26.172176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:26.174215 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:26.188202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d785f-92c0-4145-87c4-7d6a1d086161 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:26.192613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:26.193478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:26.193946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:26.194379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:26.196621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:26.265127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:26.266571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:26.275741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1238/2475] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:30.775556 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:30.776075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:30.776169 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:30.805456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] MFA Rules not processed for user `f1d4360de70b4beea3456aa3dc85d5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:30.806855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:30.885486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1238/2476] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:30.890876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:30.940649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:30.958954 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b512-b370-4e64-9e5b-6c42c65073e8 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:30.963561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:30.968047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:30.968047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:30.968047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:30.968047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:30.968047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:31.046352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1239/2477] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:30 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:31.050508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:31.062170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336107ba-a7b1-44ad-ab34-2376c67e36cf None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:31.087858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:31.121024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:31.121718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:31.122001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:31.122278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:31.123457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:31.125486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:31.197972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1239/2478] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:31 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:31.202407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:31.212961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4933f3-eed4-4179-b173-d18c8cfdab3e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:31.219572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:31.220263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:31.220612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:31.220902 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:31.221640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:31.223253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:31.238569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1240/2479] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:31.240079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:31.251518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1f379-5d8e-4cb3-9376-4c35a83a4fd3 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:31.256022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:31.257002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:31.257240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:31.257510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] PATH_INFO: `/v3/projects/a02d65db8b4d46b48a3dd131e59fe886/users/77d9fb0819ec4442bf7584680dbb4cb2/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:31.272953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02d65db8b4d46b48a3dd131e59fe886, user_id=77d9fb0819ec4442bf7584680dbb4cb2, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:31.275661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:31.349541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1240/2480] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:31 2022] PUT /identity/v3/projects/a02d65db8b4d46b48a3dd131e59fe886/users/77d9fb0819ec4442bf7584680dbb4cb2/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:31.353678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:31.365530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c8e64-bdcd-4a9a-b8f1-00c8658da509 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:31.427300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:31.428002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:31.428290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:31.428674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:31.429405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:31.431022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:31.442066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1241/2481] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:31.446995 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:31.460417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5506ee-2e09-426d-994b-7cba14cfa42e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:31.465572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb210d-c6b8-4f38-a99b-b41b07e1bb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:31.466546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb210d-c6b8-4f38-a99b-b41b07e1bb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:31.467080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb210d-c6b8-4f38-a99b-b41b07e1bb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:31.467227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb210d-c6b8-4f38-a99b-b41b07e1bb36 admin admin] PATH_INFO: `/v3/projects/a02d65db8b4d46b48a3dd131e59fe886/users/77d9fb0819ec4442bf7584680dbb4cb2/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:31.469593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb210d-c6b8-4f38-a99b-b41b07e1bb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02d65db8b4d46b48a3dd131e59fe886, user_id=77d9fb0819ec4442bf7584680dbb4cb2, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:31.472345 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb210d-c6b8-4f38-a99b-b41b07e1bb36 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:31.483966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1241/2482] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:31 2022] PUT /identity/v3/projects/a02d65db8b4d46b48a3dd131e59fe886/users/77d9fb0819ec4442bf7584680dbb4cb2/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:59:32.701800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:32.769245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564b60d-47e1-48fd-b623-a183087dae3a admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:32.850510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:32.851895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:32.851895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:32.852204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:32.891494 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:32.964224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:32.966815 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:32.975282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1242/2483] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:35.570449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:35.571539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:35.572774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:35.595430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:35.597011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:35.683583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1242/2484] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:35.849561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:35.851113 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:36.049303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33212beb-358e-42ac-b5bf-e87bba310049 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:36.058959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68efd69-e847-492e-8dd5-6270ed97b5a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:36.059623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68efd69-e847-492e-8dd5-6270ed97b5a2 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:36.060614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68efd69-e847-492e-8dd5-6270ed97b5a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:36.061412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68efd69-e847-492e-8dd5-6270ed97b5a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:36.071839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68efd69-e847-492e-8dd5-6270ed97b5a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:36.074028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68efd69-e847-492e-8dd5-6270ed97b5a2 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:36.085264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1243/2485] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 03:59:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:59:37.762825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:37.868260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10322-155d-46af-b7cc-674bb63cff5a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:37.889385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:37.890806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:37.891577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:37.892562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] PATH_INFO: `/v3/users/6c5066b6b7614a278c83f495de399d28` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:37.893978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5066b6b7614a278c83f495de399d28)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:37.895972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:38.033264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:38.153027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5066b6b7614a278c83f495de399d28'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:38.154639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5066b6b7614a278c83f495de399d28'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:38.179860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5066b6b7614a278c83f495de399d28'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:38.184720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5066b6b7614a278c83f495de399d28'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:38.259601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5066b6b7614a278c83f495de399d28'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:38.302314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1243/2486] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:37 2022] DELETE /identity/v3/users/6c5066b6b7614a278c83f495de399d28 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:38.306788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee156d-d3db-4568-8a62-63f266b6c903 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:38.308629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee156d-d3db-4568-8a62-63f266b6c903 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:38.308629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee156d-d3db-4568-8a62-63f266b6c903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:38.345074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a8ee156d-d3db-4568-8a62-63f266b6c903 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:38.346797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a8ee156d-d3db-4568-8a62-63f266b6c903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:38.476646 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1244/2487] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:38.494285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:38.496360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:38.628085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28539caf-33bf-4472-846a-ba155b8301d6 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:38.644261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:38.645002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:38.645295 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:38.648544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:38.674736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:38.676431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:38.707184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1244/2488] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:38.946668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:38.947675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:38.948111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:38.983417 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] MFA Rules not processed for user `77d9fb0819ec4442bf7584680dbb4cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:38.984718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:39.054145 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:39.106932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1245/2489] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:39.120574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:39.123647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:39.214012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2b325-71da-4370-a2fa-67a9fae3c3da service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:39.220907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:39.222642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:39.223206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:39.223456 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:39.225434 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:39.291220 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:39.323676 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:39.326508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:39.342788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1245/2490] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:39.423150 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:39.424692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:39.467865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989696da-76ac-4fb2-a44a-af64abadc528 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:39.473616 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:39.474525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:39.474828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:39.475114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] PATH_INFO: `/v3/projects/14c4f8d86eae4a3bbaec5b8e587f4d1b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:39.477313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c4f8d86eae4a3bbaec5b8e587f4d1b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:39.480151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:39.674814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c4f8d86eae4a3bbaec5b8e587f4d1b', 'request_id': 'req-d165e1bc-012a-4926-a8fc-027ed0aff36e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:39.740741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c4f8d86eae4a3bbaec5b8e587f4d1b', 'request_id': 'req-d165e1bc-012a-4926-a8fc-027ed0aff36e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:39.753380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c4f8d86eae4a3bbaec5b8e587f4d1b', 'request_id': 'req-d165e1bc-012a-4926-a8fc-027ed0aff36e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:39.773659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] The token cache is being invalidate because project 14c4f8d86eae4a3bbaec5b8e587f4d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:39.773936 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:39.775247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:39.785987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1246/2491] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:39 2022] DELETE /identity/v3/projects/14c4f8d86eae4a3bbaec5b8e587f4d1b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:39.786182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:39.791907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:39.794795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:39.805751 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ae538-5c4a-4776-8602-a4d7dde08012 service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:39.896471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:39.897489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:39.897774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:39.898053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] PATH_INFO: `/v3/users/9718af7dbe0e43bbad1a8e833484a643` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:39.899945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9718af7dbe0e43bbad1a8e833484a643)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:39.907109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:39.919173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165e1bc-012a-4926-a8fc-027ed0aff36e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:39.928304 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:39.929187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:39.929568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:39.929940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:39.937282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:40.055551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:40.102174 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:40.104108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:40.136269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1247/2492] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:40.181273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9718af7dbe0e43bbad1a8e833484a643'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:40.183917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9718af7dbe0e43bbad1a8e833484a643'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:40.208124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9718af7dbe0e43bbad1a8e833484a643'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:40.209852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9718af7dbe0e43bbad1a8e833484a643'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:40.344845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9718af7dbe0e43bbad1a8e833484a643'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:40.380224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1246/2493] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:39 2022] DELETE /identity/v3/users/9718af7dbe0e43bbad1a8e833484a643 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:41.312840 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:41.315096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:41.334742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d5ced-ec23-49fd-9993-6d766f3b647c service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:41.401746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:41.409128 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:41.409442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:41.409692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] PATH_INFO: `/v3/projects/3ca9b62811be45c884c901d6c1b8e52f` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:41.410963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca9b62811be45c884c901d6c1b8e52f)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:41.412951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:41.580738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca9b62811be45c884c901d6c1b8e52f', 'request_id': 'req-3b7f86ce-d30d-485a-9746-2ef24bdca437'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:41.772235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca9b62811be45c884c901d6c1b8e52f', 'request_id': 'req-3b7f86ce-d30d-485a-9746-2ef24bdca437'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:41.784464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca9b62811be45c884c901d6c1b8e52f', 'request_id': 'req-3b7f86ce-d30d-485a-9746-2ef24bdca437'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:41.801602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] The token cache is being invalidate because project 3ca9b62811be45c884c901d6c1b8e52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:41.802248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:41.802940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:41.808657 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1248/2494] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:41 2022] DELETE /identity/v3/projects/3ca9b62811be45c884c901d6c1b8e52f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:41.810628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:41.812463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:41.839100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945d505-6694-48b6-8f96-6ec351929a9e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:41.942554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:41.943855 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:41.944342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:41.945214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] PATH_INFO: `/v3/users/500cf044a2a441c0b8f65de758cc939e` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:41.947184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500cf044a2a441c0b8f65de758cc939e)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:41.949639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:41.955138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:42.215000 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500cf044a2a441c0b8f65de758cc939e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:42.216568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500cf044a2a441c0b8f65de758cc939e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:42.251992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500cf044a2a441c0b8f65de758cc939e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:42.252925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500cf044a2a441c0b8f65de758cc939e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:42.353331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-55af8a99-7e1d-450c-be6d-aef6a20d8e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500cf044a2a441c0b8f65de758cc939e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:42.392311 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1247/2495] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:41 2022] DELETE /identity/v3/users/500cf044a2a441c0b8f65de758cc939e => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:43.322147 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:43.324288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:43.353763 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f86ce-d30d-485a-9746-2ef24bdca437 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:43.517596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:43.518645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:43.519027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:43.519407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] PATH_INFO: `/v3/projects/20106c969af7479d8d18ced0b2150f42` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:43.520866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20106c969af7479d8d18ced0b2150f42)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:43.523262 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:43.526557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:43.663127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20106c969af7479d8d18ced0b2150f42', 'request_id': 'req-5d316718-ed3b-4c4e-b781-0126ba0f51e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:43.729280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20106c969af7479d8d18ced0b2150f42', 'request_id': 'req-5d316718-ed3b-4c4e-b781-0126ba0f51e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:43.742181 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20106c969af7479d8d18ced0b2150f42', 'request_id': 'req-5d316718-ed3b-4c4e-b781-0126ba0f51e0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:43.761776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] The token cache is being invalidate because project 20106c969af7479d8d18ced0b2150f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:43.762387 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:43.762823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-5d316718-ed3b-4c4e-b781-0126ba0f51e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:43.764460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1249/2496] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:43 2022] DELETE /identity/v3/projects/20106c969af7479d8d18ced0b2150f42 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:43.771545 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:43.771991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:43.772214 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:43.803349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:43.807013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:43.932174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1248/2497] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:43.942393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:43.942878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:43.943821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:43.980306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:43.981601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:43.991191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1250/2498] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:43.998084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:44.008382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1c1eb-cc91-4578-b712-b8b37e38564e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:44.016081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:44.016902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.017485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.018011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.018852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:59:44.019264 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:44.021128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:44.047611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1249/2499] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:59:44.048649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:44.067670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd643a1b-6a89-47b1-a983-cf2cb32661f3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:44.077844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:44.078454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.078746 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.079031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.079877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:44.083300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:44.183580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1251/2500] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:44 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:44.184837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:44.199783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62135a-2bd0-4ddc-aaeb-499b3786f18e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:44.338856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:44.339611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.340033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.340218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.340924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:44.352364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:44.433513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:44.450826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1250/2501] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:44 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:44.455952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:44.481378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340309c-4bdf-4d6a-93eb-df8691d6fa06 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:44.499569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:44.500655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.501500 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.501996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.504357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:44.506601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:44.527416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1252/2502] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:44.532660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:44.550136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f6106-4ab1-4671-8daa-f6ad202baa20 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:44.555513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db055e-6c7a-4328-8a59-ba7944cdc7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:44.556390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db055e-6c7a-4328-8a59-ba7944cdc7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.557254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db055e-6c7a-4328-8a59-ba7944cdc7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.557254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db055e-6c7a-4328-8a59-ba7944cdc7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.557956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db055e-6c7a-4328-8a59-ba7944cdc7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:44.563006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db055e-6c7a-4328-8a59-ba7944cdc7c7 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:44.582600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1251/2503] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:44.583878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:44.598155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281e4d7-973b-47a9-b3a4-c8129543b553 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:44.603025 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:44.604088 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.604528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.604826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] PATH_INFO: `/v3/projects/0746a6e421064b97a3cf099b22d1151e/users/1da074c1ed0f456c9c80e0560908ce34/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.623474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0746a6e421064b97a3cf099b22d1151e, user_id=1da074c1ed0f456c9c80e0560908ce34, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:44.628689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:44.754723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1253/2504] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:44 2022] PUT /identity/v3/projects/0746a6e421064b97a3cf099b22d1151e/users/1da074c1ed0f456c9c80e0560908ce34/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:44.768268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:44.768581 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:44.768869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:44.802151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] MFA Rules not processed for user `1da074c1ed0f456c9c80e0560908ce34`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:44.803436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:44.930289 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1252/2505] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:44.935225 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:45.061911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45622-7c13-475a-87a1-25fe510ecf04 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:45.071317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:45.072263 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:45.072813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:45.073246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:45.074126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:45.086137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:45.363955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1254/2506] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:44 2022] POST /identity/v3/projects => generated 387 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:45.368134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:45.393579 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51cde6-5441-4e7d-ab64-1077a88faa4c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:45.534120 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:45.534731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:45.535091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:45.543599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:45.543599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:45.543599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:45.666174 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1253/2507] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:45 2022] POST /identity/v3/users => generated 300 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:45.667364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:45.680984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96766857-b220-4b34-bf59-f4a8650c99b4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:45.687069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:45.687808 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:45.688144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:45.688457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:45.689251 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:45.691221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:45.698015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:45.705190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1255/2508] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:45.719430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:45.731547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f32ec0-d096-44e1-892b-e55cc8ba0b6f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:45.737940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:45.739008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:45.739354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:45.739667 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] PATH_INFO: `/v3/projects/bef54e36f5024b5baf62672e251cbaf9/users/1f95b01c4ab84594852abcca6a15ccb6/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:45.759035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef54e36f5024b5baf62672e251cbaf9, user_id=1f95b01c4ab84594852abcca6a15ccb6, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:45.761875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:46.221953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1254/2509] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:45 2022] PUT /identity/v3/projects/bef54e36f5024b5baf62672e251cbaf9/users/1f95b01c4ab84594852abcca6a15ccb6/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:46.226562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:46.238478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d3974-81a7-4263-a03d-4d796c42cf33 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:46.307766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606c14f-94f1-4fc6-a266-ca9a6ca51227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:46.311250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606c14f-94f1-4fc6-a266-ca9a6ca51227 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:46.311250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606c14f-94f1-4fc6-a266-ca9a6ca51227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:46.311250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606c14f-94f1-4fc6-a266-ca9a6ca51227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:46.311250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606c14f-94f1-4fc6-a266-ca9a6ca51227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:46.311784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606c14f-94f1-4fc6-a266-ca9a6ca51227 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:46.323199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1256/2510] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:46.327578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:46.339432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ff60b-9b9b-4304-9c07-4fea40696461 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:46.345820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:46.347061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:46.347338 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:46.347784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] PATH_INFO: `/v3/projects/bef54e36f5024b5baf62672e251cbaf9/users/1f95b01c4ab84594852abcca6a15ccb6/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:46.349955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef54e36f5024b5baf62672e251cbaf9, user_id=1f95b01c4ab84594852abcca6a15ccb6, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:46.352681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:46.375199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1255/2511] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:46 2022] PUT /identity/v3/projects/bef54e36f5024b5baf62672e251cbaf9/users/1f95b01c4ab84594852abcca6a15ccb6/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:46.377155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:46.378550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:46.378749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:46.417084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] MFA Rules not processed for user `1f95b01c4ab84594852abcca6a15ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:46.418472 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:46.512485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1257/2512] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:46.521589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:46.523601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:46.567893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:46.615878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fa246-9605-4975-894d-d5db5ef2a27c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:46.619499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2addf-b04f-47f4-adcd-1628bf55be4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:46.620191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2addf-b04f-47f4-adcd-1628bf55be4e service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:46.620472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2addf-b04f-47f4-adcd-1628bf55be4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:46.620775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2addf-b04f-47f4-adcd-1628bf55be4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:46.637732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2addf-b04f-47f4-adcd-1628bf55be4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:46.641296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2addf-b04f-47f4-adcd-1628bf55be4e service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:46.674175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1256/2513] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:59:48.036300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:48.154273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c3db-a0ae-45ce-ab96-71f97a1b4878 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:48.160572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:48.161138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:48.161785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:48.161785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] PATH_INFO: `/v3/users/1da074c1ed0f456c9c80e0560908ce34` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:48.162893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1da074c1ed0f456c9c80e0560908ce34)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:48.164953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:48.197110 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:48.373559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1da074c1ed0f456c9c80e0560908ce34'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:48.375482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1da074c1ed0f456c9c80e0560908ce34'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:48.376360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1da074c1ed0f456c9c80e0560908ce34'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:48.392288 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1da074c1ed0f456c9c80e0560908ce34'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:48.399983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1da074c1ed0f456c9c80e0560908ce34'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:48.491209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1258/2514] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:48 2022] DELETE /identity/v3/users/1da074c1ed0f456c9c80e0560908ce34 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:48.495814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:48.496443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:48.496443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:48.517737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:48.520884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:48.589612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1257/2515] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:48.601134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:48.612777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:48.681034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedba7da-41a3-408e-a274-40f1841ba9e7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:48.685701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:48.686351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:48.686632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:48.686907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:48.698479 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:48.700015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:48.711879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1259/2516] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:49.276268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:49.278177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:49.298734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2fcd3-aa2d-40eb-8ddd-658fe0d2256d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:49.311853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:49.312794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:49.313082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:49.313361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] PATH_INFO: `/v3/projects/0746a6e421064b97a3cf099b22d1151e` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:49.314853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0746a6e421064b97a3cf099b22d1151e)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:49.316683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:49.464801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0746a6e421064b97a3cf099b22d1151e', 'request_id': 'req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:49.475747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0746a6e421064b97a3cf099b22d1151e', 'request_id': 'req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:49.808099 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0746a6e421064b97a3cf099b22d1151e', 'request_id': 'req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:49.820370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] The token cache is being invalidate because project 0746a6e421064b97a3cf099b22d1151e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:49.820749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:49.821359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:49.822959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1258/2517] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:49 2022] DELETE /identity/v3/projects/0746a6e421064b97a3cf099b22d1151e => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:49.827064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:49.837373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d396c-47e5-4da3-b3dd-2ae173da1921 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:49.915691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:49.916604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:49.916892 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:49.917170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] PATH_INFO: `/v3/users/1f95b01c4ab84594852abcca6a15ccb6` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:49.918365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f95b01c4ab84594852abcca6a15ccb6)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:49.921049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:50.142591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:50.145632 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f95b01c4ab84594852abcca6a15ccb6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:50.146348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f95b01c4ab84594852abcca6a15ccb6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:50.146957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f95b01c4ab84594852abcca6a15ccb6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:50.171321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f95b01c4ab84594852abcca6a15ccb6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:50.181513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4955758f-7523-4413-863a-0873a53113a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f95b01c4ab84594852abcca6a15ccb6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:50.272959 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1260/2518] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:49 2022] DELETE /identity/v3/users/1f95b01c4ab84594852abcca6a15ccb6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:50.883293 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:50.884906 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:50.914925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c2a6-58d8-4416-970d-b3ddb8b4a189 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:50.990609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:50.998364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:50.998364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:50.998364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:50.998958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] PATH_INFO: `/v3/projects/bef54e36f5024b5baf62672e251cbaf9` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:50.999679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef54e36f5024b5baf62672e251cbaf9)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.001651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.147505 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef54e36f5024b5baf62672e251cbaf9', 'request_id': 'req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:51.152913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef54e36f5024b5baf62672e251cbaf9', 'request_id': 'req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:51.271892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef54e36f5024b5baf62672e251cbaf9', 'request_id': 'req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:51.284877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] The token cache is being invalidate because project bef54e36f5024b5baf62672e251cbaf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:51.285243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:51.285948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-82ae5ae9-4c5d-4df5-8ae8-79971440db0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:51.287539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1259/2519] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:50 2022] DELETE /identity/v3/projects/bef54e36f5024b5baf62672e251cbaf9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:51.292738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.293272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.293272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.312204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:51.313570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:51.376330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1261/2520] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:51.384191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.384485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.384795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.405967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:51.407110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:51.415957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1260/2521] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:51.421560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:51.432017 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a15b3-f675-4b0c-9070-a1ee790e1b9e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:51.436883 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:51.437468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.437756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.438112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.438802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:59:51.439074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.440788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.454228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1262/2522] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:59:51.458554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:51.468493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580ce3d-1f78-4575-a33c-778c6d976c23 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:51.473383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:51.474033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.474440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.474717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.475498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.477970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.563454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1261/2523] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:51 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:51.567631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:51.578771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5d579-159e-437b-bb3f-9494c91eff30 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:51.639919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:51.640678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.640974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.641260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.641971 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.643952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.727877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1263/2524] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:51 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:51.732207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:51.743401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836c17c-ada2-451f-87b7-67ef983fe397 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:51.749173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:51.749984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.751181 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.751461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.752199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.753839 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.765856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1262/2525] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:51.769939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:51.781458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b673f84-00fc-4018-9ea7-9f781bd77c39 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:51.786168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10884f-d8c8-43a4-bf8d-133fb50c970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:51.786843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10884f-d8c8-43a4-bf8d-133fb50c970d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.787407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10884f-d8c8-43a4-bf8d-133fb50c970d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.787705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10884f-d8c8-43a4-bf8d-133fb50c970d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.788457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10884f-d8c8-43a4-bf8d-133fb50c970d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.790094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10884f-d8c8-43a4-bf8d-133fb50c970d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.800963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1264/2526] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:51.805167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:51.815820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66a99-8fe6-4116-a12a-a40f3e7c6533 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:51.820859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:51.821749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.822038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.822317 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] PATH_INFO: `/v3/projects/7ed648380f9d4d06b543f4ce14deb012/users/0356cee247b84ee7b84310ac0a1bd272/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.840386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed648380f9d4d06b543f4ce14deb012, user_id=0356cee247b84ee7b84310ac0a1bd272, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:51.843735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:51.927978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1263/2527] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:51 2022] PUT /identity/v3/projects/7ed648380f9d4d06b543f4ce14deb012/users/0356cee247b84ee7b84310ac0a1bd272/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:51.939654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:51.939898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:51.940139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:51.971125 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] MFA Rules not processed for user `0356cee247b84ee7b84310ac0a1bd272`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:51.972345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:51.989412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:52.050960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1265/2528] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:52.058080 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.128747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4358a-b0b7-4e49-b3a6-7a4ce68b04a4 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.134525 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.137927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.137927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.137927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.137927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.139222 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.222448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1264/2529] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:52 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:52.226407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.238208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39dbba-6212-4711-86f8-ca727ba8593b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.298821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.299525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.299832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.300118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.300864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.302811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.392552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1266/2530] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:52 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:52.396437 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.406764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a752e6-dcd6-4449-9bf4-3aa541351ca0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.411728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.412436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.412761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.413052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.413857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.415497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.417089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:52.427780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1265/2531] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:52.432080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.443990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c2cd-ab96-4e15-b965-d530906a2d54 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.449072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.449976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.450260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.450536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] PATH_INFO: `/v3/projects/0cbac846f3a04eeaaf42625163dce4a5/users/70b39cd1f660499cad9092c62ed95b92/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.467428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbac846f3a04eeaaf42625163dce4a5, user_id=70b39cd1f660499cad9092c62ed95b92, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.470637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.522590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.524299 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:52.573141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1267/2532] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:52 2022] PUT /identity/v3/projects/0cbac846f3a04eeaaf42625163dce4a5/users/70b39cd1f660499cad9092c62ed95b92/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:52.578123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.588086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e26ca-4a71-4fa7-80ca-75d55cd4a243 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.656715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3a4b-011f-4ad0-a0f9-4732078534a1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.657487 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.658085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.658371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.658642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.660542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:52.662021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.662707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.663022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.663310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.664052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.665690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.671073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.672536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022877a-13ea-41b0-977b-0e3c91385bb5 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.677079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1268/2533] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:59:52.681335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.683523 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1266/2534] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:59:52.694617 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a9eb3-7eef-422d-bc01-12cf6583823e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.699488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.700390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.700816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.706071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] PATH_INFO: `/v3/projects/0cbac846f3a04eeaaf42625163dce4a5/users/70b39cd1f660499cad9092c62ed95b92/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.706071 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbac846f3a04eeaaf42625163dce4a5, user_id=70b39cd1f660499cad9092c62ed95b92, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.706284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.717868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1269/2535] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:52 2022] PUT /identity/v3/projects/0cbac846f3a04eeaaf42625163dce4a5/users/70b39cd1f660499cad9092c62ed95b92/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:52.728291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.728648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.728936 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.757191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] MFA Rules not processed for user `70b39cd1f660499cad9092c62ed95b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:52.758445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:52.828885 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1267/2536] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:52.840135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:52.842502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:52.853126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:52.909837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcce84e-e1a6-443e-9b65-cdc9ad9469be admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:52.912967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb63d60-d94a-43f5-ba5f-51d696f6305c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:52.913621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63d60-d94a-43f5-ba5f-51d696f6305c service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:52.913943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63d60-d94a-43f5-ba5f-51d696f6305c service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:52.914870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63d60-d94a-43f5-ba5f-51d696f6305c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:52.926766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb63d60-d94a-43f5-ba5f-51d696f6305c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:52.928205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb63d60-d94a-43f5-ba5f-51d696f6305c service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:52.938709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1270/2537] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:53.559546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:53.608125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:53.627444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ef3a1-824f-4ec8-b106-3cf88f698157 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:53.632178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:53.633124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:53.633408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:53.633687 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] PATH_INFO: `/v3/users/0356cee247b84ee7b84310ac0a1bd272` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:53.634910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0356cee247b84ee7b84310ac0a1bd272)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:53.636948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:53.880010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0356cee247b84ee7b84310ac0a1bd272'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:53.880641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0356cee247b84ee7b84310ac0a1bd272'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:53.889374 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0356cee247b84ee7b84310ac0a1bd272'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:53.893503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0356cee247b84ee7b84310ac0a1bd272'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:53.974171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0356cee247b84ee7b84310ac0a1bd272'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:53.994814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1268/2538] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:53 2022] DELETE /identity/v3/users/0356cee247b84ee7b84310ac0a1bd272 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:53.999619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:54.001268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:54.001268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:54.019197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:54.020460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:54.086117 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1271/2539] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:54.096896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:54.098886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:54.182528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36010f18-a172-45e7-958f-23d6a5f589ec admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:54.186644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:54.187314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:54.187763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:54.187898 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:54.198542 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:54.200094 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:54.215895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1269/2540] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:54.772533 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:54.775878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:54.789955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1027988-34d3-4ed9-a1b5-816c8ba7b3b8 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:54.794792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:54.795704 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:54.796054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:54.796333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] PATH_INFO: `/v3/projects/7ed648380f9d4d06b543f4ce14deb012` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:54.797677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed648380f9d4d06b543f4ce14deb012)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:54.801627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:54.919608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed648380f9d4d06b543f4ce14deb012', 'request_id': 'req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.008569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed648380f9d4d06b543f4ce14deb012', 'request_id': 'req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.016175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed648380f9d4d06b543f4ce14deb012', 'request_id': 'req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.026408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] The token cache is being invalidate because project 7ed648380f9d4d06b543f4ce14deb012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:55.026699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.027338 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.029001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1272/2541] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:54 2022] DELETE /identity/v3/projects/7ed648380f9d4d06b543f4ce14deb012 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:55.032935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:55.042704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2a473-f52c-4fe0-8939-3740ae6c0442 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:55.125131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:55.126296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:55.126596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:55.126870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] PATH_INFO: `/v3/users/70b39cd1f660499cad9092c62ed95b92` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:55.128115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b39cd1f660499cad9092c62ed95b92)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:55.130115 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:55.132570 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:55.321205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b39cd1f660499cad9092c62ed95b92'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.321912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b39cd1f660499cad9092c62ed95b92'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.331107 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b39cd1f660499cad9092c62ed95b92'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.331869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b39cd1f660499cad9092c62ed95b92'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.431700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-315f5fab-be74-4da1-9879-2908a9feeb5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b39cd1f660499cad9092c62ed95b92'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:55.451801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1270/2542] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:55 2022] DELETE /identity/v3/users/70b39cd1f660499cad9092c62ed95b92 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:55.999342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.000932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:56.016080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ab4cd-e93b-44e8-940a-5cf2b2f3f8c7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.051468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:56.083680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.085031 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.085390 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.085880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] PATH_INFO: `/v3/projects/0cbac846f3a04eeaaf42625163dce4a5` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.087137 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cbac846f3a04eeaaf42625163dce4a5)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.089164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:56.204612 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cbac846f3a04eeaaf42625163dce4a5', 'request_id': 'req-ec3d5761-5b70-4aee-bc5f-a346a528cffc'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:56.283064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cbac846f3a04eeaaf42625163dce4a5', 'request_id': 'req-ec3d5761-5b70-4aee-bc5f-a346a528cffc'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:56.290194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cbac846f3a04eeaaf42625163dce4a5', 'request_id': 'req-ec3d5761-5b70-4aee-bc5f-a346a528cffc'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:56.302037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] The token cache is being invalidate because project 0cbac846f3a04eeaaf42625163dce4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:56.302248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:56.302907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ec3d5761-5b70-4aee-bc5f-a346a528cffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:56.304777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1273/2543] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:55 2022] DELETE /identity/v3/projects/0cbac846f3a04eeaaf42625163dce4a5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:56.310070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.316638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.316638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.330757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:56.331986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:56.392711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1271/2544] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:56.455349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.455644 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.455930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.477100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:56.478346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:56.486944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1274/2545] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:56.491200 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.501852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f74ba-a486-46da-a67d-4e22864847ad None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.506589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.507251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.508810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.508810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.508810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:59:56.509098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.510543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:56.522852 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1272/2546] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 03:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:59:56.527045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.539376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3a349-3970-4126-8595-e719c5e6e2b0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.543742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.544525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.545801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.546230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.546943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.552904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:56.657047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1275/2547] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 03:59:56 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:59:56.661269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.672888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6be32-b83b-44ac-8834-c9ba7dda6321 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.737013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.737686 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.737966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.738280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.739027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.741002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:56.820684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:56.827781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1273/2548] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 03:59:56 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:59:56.831355 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.843021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a401b54-1ab0-4bbe-a446-4d89f5b0728e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.847858 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.848564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.848929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.849126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.849859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.851454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:56.863324 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1276/2549] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:56.867672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.881148 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02851-cfb2-4264-a5ea-73d9e2b30e91 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.885884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb00ae7-20ef-4b3b-aa7d-e0ede59fdaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.887758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00ae7-20ef-4b3b-aa7d-e0ede59fdaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.887758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00ae7-20ef-4b3b-aa7d-e0ede59fdaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.887758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb00ae7-20ef-4b3b-aa7d-e0ede59fdaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.888100 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb00ae7-20ef-4b3b-aa7d-e0ede59fdaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.890124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb00ae7-20ef-4b3b-aa7d-e0ede59fdaaf admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:56.901196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1274/2550] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 03:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:59:56.906478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:56.916653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878bc3a-d5b0-4976-b03f-945333bb0780 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:56.921749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:56.922694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:56.922981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:56.923378 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] PATH_INFO: `/v3/projects/b10025946caa45ed8a680f571722dfc3/users/d14a083b6a3242428354e1195557e8b4/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:56.938202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10025946caa45ed8a680f571722dfc3, user_id=d14a083b6a3242428354e1195557e8b4, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:56.945546 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:57.060718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1277/2551] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 03:59:56 2022] PUT /identity/v3/projects/b10025946caa45ed8a680f571722dfc3/users/d14a083b6a3242428354e1195557e8b4/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:57.071019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:57.071240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:57.071518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:57.102690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] MFA Rules not processed for user `d14a083b6a3242428354e1195557e8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:57.104560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:57.174844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1275/2552] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:57.186446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:57.190527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:57.247481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870880-5e28-404b-91e9-8bcdea633cd1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:57.251677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d0f69-5fc4-4087-9dc4-d886767834f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:57.252323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d0f69-5fc4-4087-9dc4-d886767834f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:57.252679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d0f69-5fc4-4087-9dc4-d886767834f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:57.252960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d0f69-5fc4-4087-9dc4-d886767834f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:57.261335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d0f69-5fc4-4087-9dc4-d886767834f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:57.262865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d0f69-5fc4-4087-9dc4-d886767834f4 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:57.273806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1278/2553] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:57.297031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:57.374912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1654ef6-e3f2-44e2-96b5-56f85377a200 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:57.384027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:57.385029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:57.385313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:57.385586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] PATH_INFO: `/v3/users/d14a083b6a3242428354e1195557e8b4` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:57.386825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14a083b6a3242428354e1195557e8b4)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:57.389406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:57.605395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14a083b6a3242428354e1195557e8b4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:57.605395 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14a083b6a3242428354e1195557e8b4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:57.609799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14a083b6a3242428354e1195557e8b4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:57.611104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14a083b6a3242428354e1195557e8b4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:57.689173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14a083b6a3242428354e1195557e8b4'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:57.703249 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:57.710037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1276/2554] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 03:59:57 2022] DELETE /identity/v3/users/d14a083b6a3242428354e1195557e8b4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:57.715300 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:57.715691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:57.716002 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:57.722843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:57.737851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:57.739156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:57.800123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1279/2555] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 03:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:57.809980 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:57.812127 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:57.870761 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ddc01-05da-4552-9563-61c2e22e6e6b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:57.880407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cd5d7-e5eb-4ad1-8362-45edb5f252c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:57.881178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cd5d7-e5eb-4ad1-8362-45edb5f252c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:57.881520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cd5d7-e5eb-4ad1-8362-45edb5f252c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:57.881814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cd5d7-e5eb-4ad1-8362-45edb5f252c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:57.893598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cd5d7-e5eb-4ad1-8362-45edb5f252c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:57.895063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cd5d7-e5eb-4ad1-8362-45edb5f252c1 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:57.906808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1277/2556] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:58.371593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:58.373445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:58.392847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50e1e4-717f-4191-a1d7-5cd37e7ef322 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:58.397876 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:58.398821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:58.399192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:58.399529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] PATH_INFO: `/v3/projects/b10025946caa45ed8a680f571722dfc3` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:58.401078 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10025946caa45ed8a680f571722dfc3)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:58.403235 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:58.507619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10025946caa45ed8a680f571722dfc3', 'request_id': 'req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:58.607168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10025946caa45ed8a680f571722dfc3', 'request_id': 'req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:58.614143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10025946caa45ed8a680f571722dfc3', 'request_id': 'req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:58.624939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] The token cache is being invalidate because project b10025946caa45ed8a680f571722dfc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:59:58.625211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:58.628831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c5b35af9-90b1-4946-8187-3c2b63b6e1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:59:58.630403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1280/2557] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 03:59:58 2022] DELETE /identity/v3/projects/b10025946caa45ed8a680f571722dfc3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:59:59.108709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef05055-e4ed-4b77-9bf6-8e050ef9974a None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.109690 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef05055-e4ed-4b77-9bf6-8e050ef9974a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.109972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef05055-e4ed-4b77-9bf6-8e050ef9974a None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.111386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1278/2558] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:59:59.117560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5633c02-ebe3-4aaa-b87c-b833dfa75d95 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.117852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5633c02-ebe3-4aaa-b87c-b833dfa75d95 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.118133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5633c02-ebe3-4aaa-b87c-b833dfa75d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.137560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b5633c02-ebe3-4aaa-b87c-b833dfa75d95 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:59.138791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b5633c02-ebe3-4aaa-b87c-b833dfa75d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:59.232051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1281/2559] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:59:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:59:59.245450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef05055-e4ed-4b77-9bf6-8e050ef9974a None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:59.247291 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7ef05055-e4ed-4b77-9bf6-8e050ef9974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:59.308946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef05055-e4ed-4b77-9bf6-8e050ef9974a None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:59.312939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ec703-e1c2-4f39-b8dc-eacb62d068d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc6f327db350476e9e5194719da4cbb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:59.313590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ec703-e1c2-4f39-b8dc-eacb62d068d3 service glance] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.313874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ec703-e1c2-4f39-b8dc-eacb62d068d3 service glance] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.314177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ec703-e1c2-4f39-b8dc-eacb62d068d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.324466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ec703-e1c2-4f39-b8dc-eacb62d068d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:59.325918 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ec703-e1c2-4f39-b8dc-eacb62d068d3 service glance] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:59.337157 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1279/2560] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:59.587935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6ef58-19ec-4a38-ae74-7f538d0a531a None None] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.588539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6ef58-19ec-4a38-ae74-7f538d0a531a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.588539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6ef58-19ec-4a38-ae74-7f538d0a531a None None] PATH_INFO: `/` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.589842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1282/2561] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:59:59.594710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b587d79-76a5-42ee-ba11-4fa815ebe8dd None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.594996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b587d79-76a5-42ee-ba11-4fa815ebe8dd None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.597793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b587d79-76a5-42ee-ba11-4fa815ebe8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.632335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-4b587d79-76a5-42ee-ba11-4fa815ebe8dd None None] MFA Rules not processed for user `65d7683f531a41b1adf3ceb94d32ce2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:59.633578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4b587d79-76a5-42ee-ba11-4fa815ebe8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:59.683692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1280/2562] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:59:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:59.696114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6ef58-19ec-4a38-ae74-7f538d0a531a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:59.697835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-65a6ef58-19ec-4a38-ae74-7f538d0a531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:59.710996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6ef58-19ec-4a38-ae74-7f538d0a531a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:59.714464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f30167-eda9-41c2-969d-e77c2963cf6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:59.715095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f30167-eda9-41c2-969d-e77c2963cf6d service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.715376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f30167-eda9-41c2-969d-e77c2963cf6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.715653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f30167-eda9-41c2-969d-e77c2963cf6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.723076 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f30167-eda9-41c2-969d-e77c2963cf6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:59.724484 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f30167-eda9-41c2-969d-e77c2963cf6d service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:59.732023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1283/2563] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:59:59.814656 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133881-17b8-4fec-b5ea-f01834f71adf None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.814950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133881-17b8-4fec-b5ea-f01834f71adf None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.815234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a133881-17b8-4fec-b5ea-f01834f71adf None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.816654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1281/2564] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 03:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:59:59.822666 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.823039 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.823330 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.831599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:59.843593 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] MFA Rules not processed for user `0ceb759c7b4b4292bb9763574d79254b`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:59:59.844891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:59.854352 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1284/2565] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 03:59:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:59:59.865257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a133881-17b8-4fec-b5ea-f01834f71adf None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:59.876385 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a133881-17b8-4fec-b5ea-f01834f71adf None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:59.880046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205ce57-e9c6-42c2-bdba-73348059214a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:59.880714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205ce57-e9c6-42c2-bdba-73348059214a service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:59.881001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205ce57-e9c6-42c2-bdba-73348059214a service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:59.881274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205ce57-e9c6-42c2-bdba-73348059214a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:59.892787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205ce57-e9c6-42c2-bdba-73348059214a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:59.894246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205ce57-e9c6-42c2-bdba-73348059214a service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:59.906129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1282/2566] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:59.973520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:59.974920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:00.042205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602e6aa-7714-4b8c-9b2e-8061cbaad4b7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:00.047761 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:00.048422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:00.048734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:00.049015 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:00.050806 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:00.063041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:00.064668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:00.075344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1285/2567] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 03:59:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:02.285612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c6238-418b-4def-971d-0cf2fe1abf58 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:02.286087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c6238-418b-4def-971d-0cf2fe1abf58 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:02.286182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c6238-418b-4def-971d-0cf2fe1abf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:02.304689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5b3c6238-418b-4def-971d-0cf2fe1abf58 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:02.305889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5b3c6238-418b-4def-971d-0cf2fe1abf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:02.325551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:02.361782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1283/2568] 10.222.0.58 () {58 vars in 1009 bytes} [Fri May 27 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:02.373489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:02.375553 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:02.445743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924cb2-87ff-439c-971a-b211226e78ea service placement] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:02.450793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:02.451445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:02.451766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:02.452062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:02.460400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:02.462070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:02.471229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1286/2569] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 04:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 04:00:06.956160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d3b6f-683a-498f-b3ee-acd57221d0de None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:06.956734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d3b6f-683a-498f-b3ee-acd57221d0de None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:06.956734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d3b6f-683a-498f-b3ee-acd57221d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:06.974599 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e28d3b6f-683a-498f-b3ee-acd57221d0de None None] MFA Rules not processed for user `f1d4360de70b4beea3456aa3dc85d5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:06.975959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e28d3b6f-683a-498f-b3ee-acd57221d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:07.033455 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1284/2570] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:00:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:07.406592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:07.416970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05f60-13de-48a9-a9dc-46637cae9251 service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:07.421286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:07.421946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:07.422281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:07.422692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:07.424642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:07.446683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:07.448813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:07.461944 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1287/2571] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:17.026549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=39 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:17.456432 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=71 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:24.330689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:24.331175 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:24.331281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:24.350962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] MFA Rules not processed for user `a766bce0474243d2836799b24da5b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:24.352160 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:24.359679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1285/2572] 10.222.0.58 () {58 vars in 967 bytes} [Fri May 27 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:24.372228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:24.384917 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5633e6c-4297-4183-9050-0a727f9cc971 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:24.390434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8eaac98b1b4f5db6c0b60c3f01f711', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:24.392192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:24.392192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:24.392192 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:24.402437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:24.404093 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:24.413101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1288/2573] 10.222.0.58 () {60 vars in 1384 bytes} [Fri May 27 04:00:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 04:00:29.628650 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:29.630230 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:29.647357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee5c78-0969-4555-a31e-106b0d4fe270 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:29.653143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:29.653791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:29.654075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:29.654348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:29.656186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:29.669132 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:29.671561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:29.692550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1286/2574] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:34.409476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=6 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:34.541935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:34.546139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:34.561861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db8762-7787-4e64-b26a-063dd1f73b20 service swift] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:34.568389 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:34.569086 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:34.569370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:34.569645 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:34.571486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:34.584403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:34.587807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:34.597590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1289/2575] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:37.243097 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:37.252480 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509460f-e2ab-4923-98d5-ffc029cade25 service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:37.256054 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:37.256715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:37.257004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:37.257280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:37.259039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:37.326804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:37.328959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:37.339096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1287/2576] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:37.716512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:37.718123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:37.731231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcf2d4-0075-403f-a725-33bc4cedf10a service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:37.735751 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:37.736398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:37.736718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:37.736998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:37.738879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:37.803200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:37.804678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:37.815755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1290/2577] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:41.380035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:41.382263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:41.395933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0989986-9a33-4d41-aedf-fd4a7511ce50 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:41.400406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:41.401349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:41.401637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:41.401917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] PATH_INFO: `/v3/users/bca16fe930244e12bbd12a23323d583e` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:41.420975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca16fe930244e12bbd12a23323d583e)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:41.420975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:41.649744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca16fe930244e12bbd12a23323d583e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:41.650397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca16fe930244e12bbd12a23323d583e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:41.657874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca16fe930244e12bbd12a23323d583e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:41.658504 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca16fe930244e12bbd12a23323d583e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:41.727261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-14c941c0-c277-46de-b018-21cdd87ca6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca16fe930244e12bbd12a23323d583e'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:41.746826 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1288/2578] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:00:41 2022] DELETE /identity/v3/users/bca16fe930244e12bbd12a23323d583e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:00:42.003465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.015114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3374668-31f2-46d2-a5bb-7e5fb27c6490 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.078779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.079714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.079998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.080275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] PATH_INFO: `/v3/projects/b1f668cbb60844d881dfb2af61132af7` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.091584 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f668cbb60844d881dfb2af61132af7)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.094234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.195954 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f668cbb60844d881dfb2af61132af7', 'request_id': 'req-58439b1d-4ce9-4294-b313-4c58f0605d99'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:42.257402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f668cbb60844d881dfb2af61132af7', 'request_id': 'req-58439b1d-4ce9-4294-b313-4c58f0605d99'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:42.264401 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f668cbb60844d881dfb2af61132af7', 'request_id': 'req-58439b1d-4ce9-4294-b313-4c58f0605d99'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:42.276243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] The token cache is being invalidate because project b1f668cbb60844d881dfb2af61132af7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:00:42.276551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:42.277179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-58439b1d-4ce9-4294-b313-4c58f0605d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:00:42.278734 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1291/2579] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:00:42 2022] DELETE /identity/v3/projects/b1f668cbb60844d881dfb2af61132af7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:00:42.283883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.284177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.284458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.304146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:42.305387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:42.337601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:42.369180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1289/2580] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:42.377577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.377866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.378146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.384516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:42.398540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:42.399827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:42.408843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1292/2581] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:42.413163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.423796 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a5c2b-c8a3-4ff0-afb4-b1fa964dd4f8 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.428779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.429451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.429766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.430046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.430828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:00:42.431139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.432781 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.445514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1290/2582] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:00:42.449793 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.459830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f1a3e-c420-4191-b28b-329a377d8476 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.464587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.465229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.465513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.465784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.466474 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.468799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.574934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1293/2583] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:00:42 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:00:42.579476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.589955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c20ac-b0ff-400f-9c92-097993e12160 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.649084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.649754 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.650041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.650378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.651011 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.652970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.715546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1291/2584] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:00:42 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:00:42.720062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.730555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431dbf9-3f4e-484c-91ef-516e77339d04 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.736127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.736847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.737132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.737406 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.738132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.739725 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.751399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1294/2585] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:00:42.755514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.765726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba6f5f-049a-4531-8318-59290eec812d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.770372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c866bd0-c645-4823-abb0-4750fcfc312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.771020 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c866bd0-c645-4823-abb0-4750fcfc312b admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.771307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c866bd0-c645-4823-abb0-4750fcfc312b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.771603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c866bd0-c645-4823-abb0-4750fcfc312b admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.772305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c866bd0-c645-4823-abb0-4750fcfc312b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.773939 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c866bd0-c645-4823-abb0-4750fcfc312b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.783930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1292/2586] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 04:00:42.788045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.798744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2e5ff-7729-4abd-a753-a90bbbd201ff admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:42.803373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:42.804244 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.804559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.804847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] PATH_INFO: `/v3/projects/c80fc33b8e6b4d359f432ace6c297b80/users/18d67c48b3a5445296a2069be87ebae8/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.821127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80fc33b8e6b4d359f432ace6c297b80, user_id=18d67c48b3a5445296a2069be87ebae8, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:42.823792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:42.879779 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1295/2587] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:00:42 2022] PUT /identity/v3/projects/c80fc33b8e6b4d359f432ace6c297b80/users/18d67c48b3a5445296a2069be87ebae8/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:00:42.881463 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45895ac-4008-4790-a32d-407838754a79 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:42.881759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45895ac-4008-4790-a32d-407838754a79 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:42.882041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45895ac-4008-4790-a32d-407838754a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:42.899986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d45895ac-4008-4790-a32d-407838754a79 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:42.901232 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d45895ac-4008-4790-a32d-407838754a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:42.962406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1293/2588] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:00:42.972951 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:42.975212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:43.033743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb3af0-a026-45c8-aa05-7ee7984aac7d admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:43.037449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:43.038096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:43.038385 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:43.038669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:43.048911 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:43.050356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:43.059870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1296/2589] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:48.401251 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7324626-fb02-4502-bd58-6a7ee7be3175 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:48.401813 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7324626-fb02-4502-bd58-6a7ee7be3175 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:48.401946 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7324626-fb02-4502-bd58-6a7ee7be3175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:48.442532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b7324626-fb02-4502-bd58-6a7ee7be3175 None None] MFA Rules not processed for user `18d67c48b3a5445296a2069be87ebae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:48.443798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b7324626-fb02-4502-bd58-6a7ee7be3175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:48.465602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:48.539616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1294/2590] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:00:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:48.556447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:48.557896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:48.679836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:00:48.688879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5716a0-1860-44ea-8da0-5d44872d634f service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:48.695391 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:48.696059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:48.696576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:48.696860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:48.707007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:48.708713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:48.719184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1297/2591] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:00:51.351988 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:51.352484 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:51.352712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:51.371721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] MFA Rules not processed for user `18d67c48b3a5445296a2069be87ebae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:00:51.373063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:51.381431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1295/2592] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:00:51.713146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:00:51.714692 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:51.752267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cb5a8-46d9-4870-9844-a9165c24a624 service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:00:51.756100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:00:51.756794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:00:51.757084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:00:51.757362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:00:51.759361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:00:51.770797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:00:51.772052 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:00:51.781100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1298/2593] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:00:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:01.366399 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=28 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:01.777705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=21 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:05.267594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:05.281343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1184f-2eb9-4be5-bc0f-da1472a654ae None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:05.286993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:05.288276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:05.288925 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:05.289043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:05.291076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:05.305215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:05.306832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa8984-6116-4a13-a3d2-06be24a6e9a7 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:05.318135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1296/2594] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:15.313262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=6 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:29.651934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:29.660787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ded19-8313-4ec3-8440-e5de22e53a91 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:29.666969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:29.666969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:29.666969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:29.666969 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:29.667875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:29.704249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:29.705731 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a7605-4de6-42f4-ac9a-52158f99d93c service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:29.727468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1299/2595] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:33.134239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25edca4-f10e-4d4e-8ebc-9317ea1dc85d None None] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:33.134787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25edca4-f10e-4d4e-8ebc-9317ea1dc85d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:33.134787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25edca4-f10e-4d4e-8ebc-9317ea1dc85d None None] PATH_INFO: `/` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:33.136223 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1297/2596] 10.222.0.58 () {54 vars in 921 bytes} [Fri May 27 04:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 04:01:33.166669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:33.167243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:33.167243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:33.214091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] MFA Rules not processed for user `4f611173abbd4a89b4f762571a760a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:33.215230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:33.283693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25edca4-f10e-4d4e-8ebc-9317ea1dc85d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:33.285483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b25edca4-f10e-4d4e-8ebc-9317ea1dc85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:33.285879 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1300/2597] 10.222.0.58 () {58 vars in 1018 bytes} [Fri May 27 04:01:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:33.302726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25edca4-f10e-4d4e-8ebc-9317ea1dc85d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:33.305685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:33.307590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:33.308520 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:33.309462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:33.309811 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:33.310155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] PATH_INFO: `/v3/users/3cfb66d6bc254e549d6d355c2bbb440a` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:33.311418 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cfb66d6bc254e549d6d355c2bbb440a)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:33.313450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:33.322621 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d6b1e-4312-4d60-b6cb-039ed5dc5bb6 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:33.328557 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:33.331774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:33.331774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:33.331774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:33.338923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:33.340585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:33.350712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1301/2598] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:33.527546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cfb66d6bc254e549d6d355c2bbb440a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:33.528236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cfb66d6bc254e549d6d355c2bbb440a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:33.535195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cfb66d6bc254e549d6d355c2bbb440a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:33.535856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cfb66d6bc254e549d6d355c2bbb440a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:33.593396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-81ab7405-2e1e-4d0a-a549-db006c4de45a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cfb66d6bc254e549d6d355c2bbb440a'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:33.611937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1298/2599] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:01:33 2022] DELETE /identity/v3/users/3cfb66d6bc254e549d6d355c2bbb440a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:01:33.852957 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:33.863891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80312e-2002-4403-a2e9-6434dd7bafd8 service placement] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:33.922754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:33.923630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:33.923921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:33.924199 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] PATH_INFO: `/v3/projects/26b40f0d69f44da79dfb780da80d4893` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:33.925465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b40f0d69f44da79dfb780da80d4893)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:33.927433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:33.952120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:34.024968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b40f0d69f44da79dfb780da80d4893', 'request_id': 'req-053a8b70-c953-4bff-8ad7-cfc129df4b39'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:34.090738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b40f0d69f44da79dfb780da80d4893', 'request_id': 'req-053a8b70-c953-4bff-8ad7-cfc129df4b39'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:34.097851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b40f0d69f44da79dfb780da80d4893', 'request_id': 'req-053a8b70-c953-4bff-8ad7-cfc129df4b39'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:34.108973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] The token cache is being invalidate because project 26b40f0d69f44da79dfb780da80d4893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:01:34.109259 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:34.109894 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-053a8b70-c953-4bff-8ad7-cfc129df4b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:34.111435 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1302/2600] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:01:33 2022] DELETE /identity/v3/projects/26b40f0d69f44da79dfb780da80d4893 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:34.116379 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.116700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.116978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.163944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:34.165905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:34.224362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1299/2601] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:34.233282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.233773 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.233874 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.252123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:34.253351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:34.261286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1303/2602] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:34.265511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.275809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0e32-508d-45e5-b5c9-f05dfc0e0d8e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.280440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.281145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.281416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.281698 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.282426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:01:34.282739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.284357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.296492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1300/2603] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 04:01:34.300946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.310127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf1b2c-c6c6-4ac1-a480-b7e7fafd3859 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.314718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.315360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.315896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.316167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.316896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.318791 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.394209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1304/2604] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:01:34 2022] POST /identity/v3/projects => generated 401 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:34.398577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.409802 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03816-4366-498d-a901-97a7ce771747 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.442335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:34.467646 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.468319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.468634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.468913 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.469622 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.471530 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.560139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1301/2605] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:01:34 2022] POST /identity/v3/users => generated 301 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:34.564443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.575771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcd6e3-061a-4aee-b730-2368258e42ee admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.580363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.581062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.581350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.581648 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.582367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.583958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.595967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1305/2606] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:34.600007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.610030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d2e9c-eb84-4aa5-afc7-7657590a5d0b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.614693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417c40f-f0d2-4c87-832a-c9069fc8b477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.615343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417c40f-f0d2-4c87-832a-c9069fc8b477 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.615630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417c40f-f0d2-4c87-832a-c9069fc8b477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.615910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417c40f-f0d2-4c87-832a-c9069fc8b477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.616668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417c40f-f0d2-4c87-832a-c9069fc8b477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.618279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417c40f-f0d2-4c87-832a-c9069fc8b477 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.628089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1302/2607] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:34.632177 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.642723 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd6e5-2fff-4a35-bebe-7a8c6a6126c4 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.647364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.648167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.648454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.648769 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] PATH_INFO: `/v3/projects/f7459e55914c4878a1b81665297a7754/users/e3ac1abf49af4467b3ece8b7242ab8bd/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.663885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7459e55914c4878a1b81665297a7754, user_id=e3ac1abf49af4467b3ece8b7242ab8bd, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.666719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.725118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1306/2608] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:34 2022] PUT /identity/v3/projects/f7459e55914c4878a1b81665297a7754/users/e3ac1abf49af4467b3ece8b7242ab8bd/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:34.730573 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b98ad-408e-46ce-a70a-666a00f43a1a None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.732719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b98ad-408e-46ce-a70a-666a00f43a1a None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.732719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b98ad-408e-46ce-a70a-666a00f43a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.750638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2e9b98ad-408e-46ce-a70a-666a00f43a1a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:34.751848 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2e9b98ad-408e-46ce-a70a-666a00f43a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:34.810789 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1303/2609] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:34.820780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:34.823208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:34.880552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f4d95-2b5d-4417-9ad2-2ce3984f74b0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:34.884629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:34.885271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:34.885555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:34.885830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:34.895016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:34.896452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:34.906276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1307/2610] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:36.342590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:36.343639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:36.343639 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:36.372832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] MFA Rules not processed for user `e3ac1abf49af4467b3ece8b7242ab8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:36.374063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:36.447010 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1304/2611] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:36.451506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:36.462256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9018f5-36a2-470b-8dd0-e7add2ac3ba2 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:36.467186 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:36.467839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:36.468124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:36.468405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:36.469131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:36.471060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:36.582482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1308/2612] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:01:36 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:36.583805 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:36.595889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ddf8-2b20-4e8e-89ca-8721891e2aa0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:36.647028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:36.661348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:36.662032 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:36.662320 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:36.662601 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:36.663305 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:36.665354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:36.753793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1305/2613] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:01:36 2022] POST /identity/v3/users => generated 308 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:36.758589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:36.769732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3853f43-74ec-487b-a167-55d737cd18c0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:36.774363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:36.774988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:36.775274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:36.775578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:36.776334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:36.777963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:36.788834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1309/2614] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:36.793260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:36.804071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b9155-03eb-486e-aa2b-fab07712b6c5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:36.809606 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:36.810634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:36.810921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:36.811194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] PATH_INFO: `/v3/projects/fabdbbc2770f4cd9aed5cddc0508b47b/users/8d9f7feaa10147fa9326ea7656f3bf2f/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:36.827586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabdbbc2770f4cd9aed5cddc0508b47b, user_id=8d9f7feaa10147fa9326ea7656f3bf2f, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:36.831678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:36.908206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1306/2615] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:36 2022] PUT /identity/v3/projects/fabdbbc2770f4cd9aed5cddc0508b47b/users/8d9f7feaa10147fa9326ea7656f3bf2f/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:01:36.912254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:36.922499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e36c6a-0395-4055-92cd-87229dd639f9 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:36.934462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:36.979051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719925bf-1261-4fc3-ba9a-1685412618f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:36.979721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719925bf-1261-4fc3-ba9a-1685412618f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:36.980860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719925bf-1261-4fc3-ba9a-1685412618f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:36.981091 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719925bf-1261-4fc3-ba9a-1685412618f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:36.981827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719925bf-1261-4fc3-ba9a-1685412618f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:36.983544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719925bf-1261-4fc3-ba9a-1685412618f4 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:36.993420 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1310/2616] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:36.999302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:37.009116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670fc62-8e09-4824-a496-0f0318133874 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:37.013950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:37.014838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:37.015119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:37.015398 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] PATH_INFO: `/v3/projects/fabdbbc2770f4cd9aed5cddc0508b47b/users/8d9f7feaa10147fa9326ea7656f3bf2f/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:37.017390 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabdbbc2770f4cd9aed5cddc0508b47b, user_id=8d9f7feaa10147fa9326ea7656f3bf2f, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:37.020037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:37.030832 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1307/2617] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:36 2022] PUT /identity/v3/projects/fabdbbc2770f4cd9aed5cddc0508b47b/users/8d9f7feaa10147fa9326ea7656f3bf2f/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:38.298868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:38.299428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:38.299428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:38.330428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] MFA Rules not processed for user `8d9f7feaa10147fa9326ea7656f3bf2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:38.331850 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:38.406114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1311/2618] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:39.743342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:39.813329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f7f59-5daf-4ff4-ad4a-6b1711a5ab8c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:39.818147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:39.819017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:39.819298 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:39.819572 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] PATH_INFO: `/v3/users/e3ac1abf49af4467b3ece8b7242ab8bd` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:39.820774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ac1abf49af4467b3ece8b7242ab8bd)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:39.822692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:40.003007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ac1abf49af4467b3ece8b7242ab8bd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.003136 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ac1abf49af4467b3ece8b7242ab8bd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.010849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ac1abf49af4467b3ece8b7242ab8bd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.012788 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ac1abf49af4467b3ece8b7242ab8bd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.101985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ac1abf49af4467b3ece8b7242ab8bd'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.121449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1308/2619] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:01:39 2022] DELETE /identity/v3/users/e3ac1abf49af4467b3ece8b7242ab8bd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:40.372358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:40.382893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e573d-c7b5-4b0c-b86d-7063f0af58a3 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:40.442045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:40.442914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:40.443204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:40.443490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] PATH_INFO: `/v3/projects/f7459e55914c4878a1b81665297a7754` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:40.444787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7459e55914c4878a1b81665297a7754)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:40.446899 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:40.547603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:40.554688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7459e55914c4878a1b81665297a7754', 'request_id': 'req-db754b9b-e718-4169-ba90-61f26a76ecf2'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.644826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7459e55914c4878a1b81665297a7754', 'request_id': 'req-db754b9b-e718-4169-ba90-61f26a76ecf2'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.651552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7459e55914c4878a1b81665297a7754', 'request_id': 'req-db754b9b-e718-4169-ba90-61f26a76ecf2'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.661966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] The token cache is being invalidate because project f7459e55914c4878a1b81665297a7754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:01:40.662250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.662881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.664419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1312/2620] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:01:40 2022] DELETE /identity/v3/projects/f7459e55914c4878a1b81665297a7754 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:40.669281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:40.670723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:40.684225 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc50de-ee6b-47dd-b3f0-142e2504b97a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:40.696722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:40.740233 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:40.741277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:40.747021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:40.747021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] PATH_INFO: `/v3/users/8d9f7feaa10147fa9326ea7656f3bf2f` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:40.747021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9f7feaa10147fa9326ea7656f3bf2f)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:40.747021 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:40.928050 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9f7feaa10147fa9326ea7656f3bf2f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.928736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9f7feaa10147fa9326ea7656f3bf2f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.936171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9f7feaa10147fa9326ea7656f3bf2f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:40.936815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9f7feaa10147fa9326ea7656f3bf2f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.022121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-8eb5e818-3039-4d6b-8f35-eed266088ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9f7feaa10147fa9326ea7656f3bf2f'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.040278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1309/2621] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:01:40 2022] DELETE /identity/v3/users/8d9f7feaa10147fa9326ea7656f3bf2f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:41.549690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:41.551245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:41.565967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db754b9b-e718-4169-ba90-61f26a76ecf2 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:41.627735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:41.628642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:41.628925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:41.629205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] PATH_INFO: `/v3/projects/fabdbbc2770f4cd9aed5cddc0508b47b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:41.630436 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabdbbc2770f4cd9aed5cddc0508b47b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:41.632348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:41.736649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabdbbc2770f4cd9aed5cddc0508b47b', 'request_id': 'req-309b8883-9b7b-44b9-9c98-51f8581ec8f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.810953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabdbbc2770f4cd9aed5cddc0508b47b', 'request_id': 'req-309b8883-9b7b-44b9-9c98-51f8581ec8f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.818350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabdbbc2770f4cd9aed5cddc0508b47b', 'request_id': 'req-309b8883-9b7b-44b9-9c98-51f8581ec8f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.830224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] The token cache is being invalidate because project fabdbbc2770f4cd9aed5cddc0508b47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:01:41.830543 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.831200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-309b8883-9b7b-44b9-9c98-51f8581ec8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:41.833046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1313/2622] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:01:41 2022] DELETE /identity/v3/projects/fabdbbc2770f4cd9aed5cddc0508b47b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:41.838280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:41.838748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:41.838870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:41.859155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:41.860540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:41.925746 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1310/2623] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:41.933473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:41.933765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:41.934146 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:41.953195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:41.954392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:41.962749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1314/2624] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:41.967823 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:41.977827 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfcc2c-b117-4274-ac5d-935c60f8a52f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:41.982865 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:41.983607 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:41.983806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:41.984086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:41.984870 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:01:41.985207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:41.986841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:41.998749 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1311/2625] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:01:42.003260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.013126 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf9f2-80d3-45e3-b90d-3b1447a69d9b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.017771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.018419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.018702 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.018977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.019673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.021615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.109631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1315/2626] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:01:42 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:42.113644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.124026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4679d-7ff2-4405-bf47-e5a9b3f80e1c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.145755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:42.183612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.184537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.184830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.185113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.185818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.187818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.256577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1312/2627] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:01:42 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:42.260924 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.272296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4562874-fc9e-4f32-804d-dc8e8d687875 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.276898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.277579 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.277881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.278162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.278882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.280476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.292309 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1316/2628] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:42.296346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.306738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568d62f-f313-42a9-875b-054938566853 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.311901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f97a37-a689-4569-83ff-37d0d0729f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.312838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f97a37-a689-4569-83ff-37d0d0729f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.313122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f97a37-a689-4569-83ff-37d0d0729f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.313396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f97a37-a689-4569-83ff-37d0d0729f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.314108 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f97a37-a689-4569-83ff-37d0d0729f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.315958 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f97a37-a689-4569-83ff-37d0d0729f0c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.326258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1313/2629] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:42.330372 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.340187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0c720-6cde-4f77-89b3-7ad77ad2ebc0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.344776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.345655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.345943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.346221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] PATH_INFO: `/v3/projects/7e29ab68a89b4adc9ec85c848c71a6e1/users/101ca61d41434f13b0e017a8a15106f0/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.360831 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e29ab68a89b4adc9ec85c848c71a6e1, user_id=101ca61d41434f13b0e017a8a15106f0, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.364308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.367431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:42.416201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1317/2630] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:42 2022] PUT /identity/v3/projects/7e29ab68a89b4adc9ec85c848c71a6e1/users/101ca61d41434f13b0e017a8a15106f0/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:42.426642 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.426937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.427224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.454718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] MFA Rules not processed for user `101ca61d41434f13b0e017a8a15106f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:42.455987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:42.519842 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1314/2631] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:42.524345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.589431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a43a-169d-4bd2-9285-84710ce6b3e8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.594783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.595449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.595736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.596041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.596787 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.598679 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.687016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1318/2632] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:01:42 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:42.692186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.705396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808629a-8f9a-4d69-b72b-ad171157500c None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.764235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.764941 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.765227 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.765508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.766195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.768110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.833728 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1315/2633] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:01:42 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:42.837796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.848302 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d20e-d242-45cd-abae-ff64eb15f30e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.853028 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.853685 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.853975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.854257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.854983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.857250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.867741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1319/2634] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:42.872829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.882944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111faedd-40fa-4ad1-aadd-2690d6bb70c4 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:42.887645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:42.888541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:42.888830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:42.889111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] PATH_INFO: `/v3/projects/8775f04616df4a32a5d7fdc0e7fb16fe/users/6049c92ead364b558b908849d46d0750/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:42.903810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8775f04616df4a32a5d7fdc0e7fb16fe, user_id=6049c92ead364b558b908849d46d0750, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:42.906492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:42.965892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:42.971762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1316/2635] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:42 2022] PUT /identity/v3/projects/8775f04616df4a32a5d7fdc0e7fb16fe/users/6049c92ead364b558b908849d46d0750/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:01:42.975828 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:42.987478 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7ce60-d4db-4a87-91dd-482737b43fca admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:43.050024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cf1dd-dc26-4f73-875f-d73a1740a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:43.050697 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf1dd-dc26-4f73-875f-d73a1740a047 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:43.050990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf1dd-dc26-4f73-875f-d73a1740a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:43.051273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cf1dd-dc26-4f73-875f-d73a1740a047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:43.052027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cf1dd-dc26-4f73-875f-d73a1740a047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:43.053626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cf1dd-dc26-4f73-875f-d73a1740a047 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:43.064188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1320/2636] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:43.068917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:43.079637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1074-3b5e-4048-8f2e-68c08baa2be2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:43.084253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:43.085163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:43.085442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:43.085714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] PATH_INFO: `/v3/projects/8775f04616df4a32a5d7fdc0e7fb16fe/users/6049c92ead364b558b908849d46d0750/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:43.087679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8775f04616df4a32a5d7fdc0e7fb16fe, user_id=6049c92ead364b558b908849d46d0750, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:43.090349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:43.100732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1317/2637] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:43 2022] PUT /identity/v3/projects/8775f04616df4a32a5d7fdc0e7fb16fe/users/6049c92ead364b558b908849d46d0750/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:43.112735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:43.113035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:43.113320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:43.141379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] MFA Rules not processed for user `6049c92ead364b558b908849d46d0750`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:43.142611 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:43.194267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:43.213703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1321/2638] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:43.225258 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:43.226981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:43.316231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a487-a6ca-49b0-80e9-13a67f44abf4 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:43.327383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ec2a6-4505-421f-ab84-f4d4617e96a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:43.328044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ec2a6-4505-421f-ab84-f4d4617e96a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:43.328366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ec2a6-4505-421f-ab84-f4d4617e96a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:43.328730 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ec2a6-4505-421f-ab84-f4d4617e96a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:43.338321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ec2a6-4505-421f-ab84-f4d4617e96a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:43.339774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ec2a6-4505-421f-ab84-f4d4617e96a4 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:43.349769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1318/2639] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:44.329801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:44.392601 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eb0ff-8b09-46b5-801d-1eea8dec64e5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:44.397059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:44.397927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:44.398213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:44.398488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] PATH_INFO: `/v3/users/101ca61d41434f13b0e017a8a15106f0` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:44.399672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101ca61d41434f13b0e017a8a15106f0)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:44.401637 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:44.606306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101ca61d41434f13b0e017a8a15106f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:44.607069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101ca61d41434f13b0e017a8a15106f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:44.607672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101ca61d41434f13b0e017a8a15106f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:44.619284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101ca61d41434f13b0e017a8a15106f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:44.626833 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101ca61d41434f13b0e017a8a15106f0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:44.736253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1322/2640] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:01:44 2022] DELETE /identity/v3/users/101ca61d41434f13b0e017a8a15106f0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:44.740773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:44.741256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:44.741384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:44.759257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:44.760439 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:44.819955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1319/2641] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:44.830807 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:44.832274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:44.894782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11697157-3dd3-40da-b1e3-13f918ca7853 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:44.898337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:44.898981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:44.899269 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:44.899550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:44.908703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:44.910130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:44.920206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1323/2642] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:45.382971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:45.384432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:45.397279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f3094-3eeb-437b-9cfd-456a17a17fa0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:45.401987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:45.402859 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:45.403149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:45.403915 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] PATH_INFO: `/v3/projects/7e29ab68a89b4adc9ec85c848c71a6e1` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:45.405206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e29ab68a89b4adc9ec85c848c71a6e1)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:45.407204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:45.537111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e29ab68a89b4adc9ec85c848c71a6e1', 'request_id': 'req-934f0a35-655d-47b7-badc-8b028b4528fc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.543685 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e29ab68a89b4adc9ec85c848c71a6e1', 'request_id': 'req-934f0a35-655d-47b7-badc-8b028b4528fc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.548119 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:45.621205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e29ab68a89b4adc9ec85c848c71a6e1', 'request_id': 'req-934f0a35-655d-47b7-badc-8b028b4528fc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.636577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] The token cache is being invalidate because project 7e29ab68a89b4adc9ec85c848c71a6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:01:45.636577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.636577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.636577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1320/2643] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:01:45 2022] DELETE /identity/v3/projects/7e29ab68a89b4adc9ec85c848c71a6e1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:45.640139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:45.650501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65fa56-d7b3-4c06-a1c5-2fd102888c7f service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:45.659369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:45.710561 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:45.711428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:45.711709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:45.711986 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] PATH_INFO: `/v3/users/6049c92ead364b558b908849d46d0750` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:45.713184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6049c92ead364b558b908849d46d0750)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:45.715103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:45.875201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6049c92ead364b558b908849d46d0750'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.875875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6049c92ead364b558b908849d46d0750'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.876535 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6049c92ead364b558b908849d46d0750'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.891032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6049c92ead364b558b908849d46d0750'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.897204 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c1bb0bd0-af09-46ed-80d2-03214fbec557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6049c92ead364b558b908849d46d0750'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:45.958990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1324/2644] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:01:45 2022] DELETE /identity/v3/users/6049c92ead364b558b908849d46d0750 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:46.327282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:46.328914 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:46.343339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f0a35-655d-47b7-badc-8b028b4528fc admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:46.402838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:46.403862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:46.404155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:46.404440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] PATH_INFO: `/v3/projects/8775f04616df4a32a5d7fdc0e7fb16fe` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:46.405665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8775f04616df4a32a5d7fdc0e7fb16fe)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:46.407616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:46.490901 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8775f04616df4a32a5d7fdc0e7fb16fe', 'request_id': 'req-c9bbe589-a713-4c87-bb0c-7928bd93f694'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:46.496661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8775f04616df4a32a5d7fdc0e7fb16fe', 'request_id': 'req-c9bbe589-a713-4c87-bb0c-7928bd93f694'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:46.559033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8775f04616df4a32a5d7fdc0e7fb16fe', 'request_id': 'req-c9bbe589-a713-4c87-bb0c-7928bd93f694'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:46.572261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] The token cache is being invalidate because project 8775f04616df4a32a5d7fdc0e7fb16fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:01:46.572580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:46.573271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-c9bbe589-a713-4c87-bb0c-7928bd93f694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:01:46.574956 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1321/2645] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:01:46 2022] DELETE /identity/v3/projects/8775f04616df4a32a5d7fdc0e7fb16fe => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:46.580046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:46.580395 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:46.580715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:46.600301 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:46.601574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:46.661281 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1325/2646] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:46.670551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:46.670871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:46.671154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:46.690983 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:46.692208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:46.700807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1322/2647] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:46.705341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:46.714967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dfdd1-c7de-41be-bdb8-a83720197f7a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:46.720757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:46.721412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:46.721698 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:46.721980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:46.722706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:01:46.723022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:46.724655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:46.737482 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1326/2648] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 04:01:46.742086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:46.752596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e698d-f7fb-4176-b909-8ce9d6b0083d None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:46.757341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:46.757992 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:46.758271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:46.758543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:46.759259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:46.761188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:46.830735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1323/2649] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:01:46 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:46.835253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:46.845384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685d41a-f94f-425c-a853-6f01b5700e27 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:46.857915 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:46.905082 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:46.905760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:46.906047 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:46.906325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:46.907013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:46.908968 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.017673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1327/2650] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:01:46 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:47.021774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.032246 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8d735-8f74-4797-bf19-7a5857db5e9e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.036778 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.037454 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.037734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.038007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.038726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.040370 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.050720 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1324/2651] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:47.054812 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.065790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e314d-0d21-4116-97ea-21f01738f982 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.070394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d2d52-fc13-4abe-abd3-6d3635c84350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.071053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2d52-fc13-4abe-abd3-6d3635c84350 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.071336 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2d52-fc13-4abe-abd3-6d3635c84350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.071615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d2d52-fc13-4abe-abd3-6d3635c84350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.072349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d2d52-fc13-4abe-abd3-6d3635c84350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.073985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d2d52-fc13-4abe-abd3-6d3635c84350 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.085513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1328/2652] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:47.089700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.099731 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30a2e-020b-4728-b252-da11cc151dab admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.104471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.105423 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.105709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.105989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] PATH_INFO: `/v3/projects/dd205b6ab700474eaa2979877f3e4cc5/users/c684f06b81b649608b53c10f65d484b6/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.121597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd205b6ab700474eaa2979877f3e4cc5, user_id=c684f06b81b649608b53c10f65d484b6, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.125059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.204903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1325/2653] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:47 2022] PUT /identity/v3/projects/dd205b6ab700474eaa2979877f3e4cc5/users/c684f06b81b649608b53c10f65d484b6/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:47.214961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.215337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.215626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.243492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] MFA Rules not processed for user `c684f06b81b649608b53c10f65d484b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:47.244738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:47.310880 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1329/2654] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:47.315544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.334856 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:47.377527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cb572-b4c8-4fc5-816d-46ed233ddb8e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.382303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.388798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.388798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.388798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.388798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.388798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.480451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1326/2655] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:01:47 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:47.484943 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.496005 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f88363-73fb-42a6-a021-4dfc83c74973 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.552834 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.553512 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.553799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.554079 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.554785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.556728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.639625 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1330/2656] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:01:47 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:01:47.643645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.654808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e777a-514a-473b-839d-9ec13ca70889 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.659451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.660179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.660462 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.660762 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.661475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.663096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.674133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1327/2657] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:01:47.678388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.689422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78ea9-4abf-45fb-9e38-ed07967e8cba admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.694310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.695216 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.695505 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.695785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] PATH_INFO: `/v3/projects/fc4235e37f6c43dc843fa6ec90845f2f/users/13efc0ce2d4d4a25bc5ea37c51be6cc6/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.708447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:47.711613 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4235e37f6c43dc843fa6ec90845f2f, user_id=13efc0ce2d4d4a25bc5ea37c51be6cc6, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.714320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.817459 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1331/2658] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:47 2022] PUT /identity/v3/projects/fc4235e37f6c43dc843fa6ec90845f2f/users/13efc0ce2d4d4a25bc5ea37c51be6cc6/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:01:47.821593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.832486 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135420db-7815-4a88-9145-b414fad89a4d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.889929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30a653-7410-465e-8f21-baa0a69cfa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.890543 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30a653-7410-465e-8f21-baa0a69cfa9b admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.890830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30a653-7410-465e-8f21-baa0a69cfa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.891103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30a653-7410-465e-8f21-baa0a69cfa9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.891841 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30a653-7410-465e-8f21-baa0a69cfa9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.893445 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30a653-7410-465e-8f21-baa0a69cfa9b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.903215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1328/2659] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 04:01:47.907527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:47.918408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228162c-bc9f-44ab-a349-e02c808e91cc admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:47.924187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:47.925106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.925397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.925678 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] PATH_INFO: `/v3/projects/fc4235e37f6c43dc843fa6ec90845f2f/users/13efc0ce2d4d4a25bc5ea37c51be6cc6/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.927650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4235e37f6c43dc843fa6ec90845f2f, user_id=13efc0ce2d4d4a25bc5ea37c51be6cc6, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:47.930364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:47.942345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1332/2660] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:01:47 2022] PUT /identity/v3/projects/fc4235e37f6c43dc843fa6ec90845f2f/users/13efc0ce2d4d4a25bc5ea37c51be6cc6/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:01:47.954757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:47.955048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:47.955329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:47.983180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] MFA Rules not processed for user `13efc0ce2d4d4a25bc5ea37c51be6cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:01:47.984352 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:48.052004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1329/2661] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:01:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:01:48.063433 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:48.065504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:01:48.120984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e442c-ac8e-4307-874c-a063dcfe66f3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:48.125233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:48.125869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:48.126148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:48.126421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:48.180272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:48.181733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:48.191414 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1333/2662] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:55.556440 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:01:55.567627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b6c7d-e9f6-4d59-8ada-8c9b0a200402 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:01:55.571206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cf4b3-62cf-4760-aa68-b558793d1763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:01:55.571849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cf4b3-62cf-4760-aa68-b558793d1763 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:01:55.572133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cf4b3-62cf-4760-aa68-b558793d1763 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:01:55.572409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cf4b3-62cf-4760-aa68-b558793d1763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:01:55.581739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cf4b3-62cf-4760-aa68-b558793d1763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:01:55.583173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cf4b3-62cf-4760-aa68-b558793d1763 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:01:55.588231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:01:55.593077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1330/2663] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:01:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:01:58.187189 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:26.754041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:26.815933 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088a45-6c55-4a4f-8e79-7022d2730c4a service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:26.820789 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:26.821434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:26.821721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:26.822001 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] PATH_INFO: `/v3/users/c684f06b81b649608b53c10f65d484b6` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:26.823183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c684f06b81b649608b53c10f65d484b6)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:26.825209 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:26.999932 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c684f06b81b649608b53c10f65d484b6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.000568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c684f06b81b649608b53c10f65d484b6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.001171 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c684f06b81b649608b53c10f65d484b6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.013520 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c684f06b81b649608b53c10f65d484b6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.021184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c684f06b81b649608b53c10f65d484b6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.106200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1334/2664] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:02:26 2022] DELETE /identity/v3/users/c684f06b81b649608b53c10f65d484b6 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:27.110780 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:27.111260 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:27.111363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:27.129726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:27.130966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:27.190426 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1331/2665] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:27.201038 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:27.202460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:27.261985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197fd88-d700-4609-ae43-f70bef390f22 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:27.265525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:27.266175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:27.266465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:27.266749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:27.276442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:27.277907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:27.287278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1335/2666] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:27.717993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:27.724155 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:27.734765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed4574-ea6d-4b20-9a17-8c0355ea78b0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:27.740210 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:27.742585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:27.742874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:27.743207 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] PATH_INFO: `/v3/projects/dd205b6ab700474eaa2979877f3e4cc5` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:27.744364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd205b6ab700474eaa2979877f3e4cc5)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:27.746506 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:27.854250 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd205b6ab700474eaa2979877f3e4cc5', 'request_id': 'req-66d47db3-a36c-47ad-812b-96ee7347ea12'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.859513 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd205b6ab700474eaa2979877f3e4cc5', 'request_id': 'req-66d47db3-a36c-47ad-812b-96ee7347ea12'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.928072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd205b6ab700474eaa2979877f3e4cc5', 'request_id': 'req-66d47db3-a36c-47ad-812b-96ee7347ea12'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.940887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] The token cache is being invalidate because project dd205b6ab700474eaa2979877f3e4cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:02:27.943029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.943029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:27.943414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1332/2667] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:02:27 2022] DELETE /identity/v3/projects/dd205b6ab700474eaa2979877f3e4cc5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:27.947539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:27.958116 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53967e49-de4e-4bb3-b6e8-f1506fe092dd service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:27.975985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:28.015268 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:28.016143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:28.016439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:28.016747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] PATH_INFO: `/v3/users/13efc0ce2d4d4a25bc5ea37c51be6cc6` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:28.018752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13efc0ce2d4d4a25bc5ea37c51be6cc6)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:28.020884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:28.223201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13efc0ce2d4d4a25bc5ea37c51be6cc6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:28.223865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13efc0ce2d4d4a25bc5ea37c51be6cc6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:28.224473 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13efc0ce2d4d4a25bc5ea37c51be6cc6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:28.238056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13efc0ce2d4d4a25bc5ea37c51be6cc6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:28.244891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4ee17cd3-eb38-449b-a275-57e670d0ef35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13efc0ce2d4d4a25bc5ea37c51be6cc6'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:28.305231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1336/2668] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:02:27 2022] DELETE /identity/v3/users/13efc0ce2d4d4a25bc5ea37c51be6cc6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:28.741396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:28.742854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:28.755964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d47db3-a36c-47ad-812b-96ee7347ea12 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:28.822019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:28.822887 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:28.823176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:28.823479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] PATH_INFO: `/v3/projects/fc4235e37f6c43dc843fa6ec90845f2f` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:28.824783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4235e37f6c43dc843fa6ec90845f2f)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:28.826753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:28.946714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:28.963700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4235e37f6c43dc843fa6ec90845f2f', 'request_id': 'req-f451143a-76ca-4664-812f-9d211462bdea'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:28.969087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4235e37f6c43dc843fa6ec90845f2f', 'request_id': 'req-f451143a-76ca-4664-812f-9d211462bdea'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:29.060649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4235e37f6c43dc843fa6ec90845f2f', 'request_id': 'req-f451143a-76ca-4664-812f-9d211462bdea'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:29.073372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] The token cache is being invalidate because project fc4235e37f6c43dc843fa6ec90845f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:02:29.073661 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:29.074309 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-f451143a-76ca-4664-812f-9d211462bdea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:29.075888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1333/2669] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:02:28 2022] DELETE /identity/v3/projects/fc4235e37f6c43dc843fa6ec90845f2f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:29.081835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.082426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.082426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.101908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:29.103130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:29.164739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1337/2670] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:29.172770 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.173062 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.173378 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.191923 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:29.193279 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:29.201144 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1334/2671] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:29.205946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.215770 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96460fb-9b61-430c-a1a2-fd17ae64bd8e None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.220349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.221841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.222159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.222443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.223176 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:02:29.223501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.225160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:29.237365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1338/2672] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:02:29.242234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.251580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e2489-1692-4688-9dbb-5991bb37a1bf None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.257135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.257783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.258067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.258333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.259003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.260940 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:29.363849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1335/2673] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:29 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:29.368129 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.378765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a4fa8-ab43-4522-953e-99f83a34be9c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.405622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:29.437608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.438278 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.438821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.439591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.440530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.442445 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:29.558152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1339/2674] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:29 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:29.562562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.574432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b8c7b-82bc-40c9-aed5-176dfceb4490 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.579013 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.579708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.580900 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.581191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.581903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.583497 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:29.593933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1336/2675] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:29.598105 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.608341 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e256a0-f795-446e-a7f8-759ed12b7764 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.612993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce987ce5-ef39-4b3e-9bec-c9fd411976e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.613653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987ce5-ef39-4b3e-9bec-c9fd411976e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.613997 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987ce5-ef39-4b3e-9bec-c9fd411976e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.614642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce987ce5-ef39-4b3e-9bec-c9fd411976e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.615421 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987ce5-ef39-4b3e-9bec-c9fd411976e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.617032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce987ce5-ef39-4b3e-9bec-c9fd411976e9 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:29.627732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1340/2676] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:29.631886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.641991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467705a-7836-439d-a714-b5cc208b3a4d admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.647387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.648268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.648587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.648869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] PATH_INFO: `/v3/projects/c737a4c54dc94f4196be29e3e6a82bf5/users/705403ae0e644b49858f2e86e00014f6/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.663846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=c737a4c54dc94f4196be29e3e6a82bf5, user_id=705403ae0e644b49858f2e86e00014f6, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.666546 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:29.741493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1337/2677] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:29 2022] PUT /identity/v3/projects/c737a4c54dc94f4196be29e3e6a82bf5/users/705403ae0e644b49858f2e86e00014f6/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:29.753481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.753777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.754061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.780122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] MFA Rules not processed for user `705403ae0e644b49858f2e86e00014f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:29.781422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:29.847273 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1341/2678] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:29.851911 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:29.914360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac552c5-e9db-4187-b482-6afc12719ced admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:29.918990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:29.919648 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:29.919930 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:29.920204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:29.920916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:29.922819 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.011397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1338/2679] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:29 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:30.017429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.026976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf50d6-0a12-42fe-8a52-a7bf8a8dd15b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.085367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.086087 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.086384 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.086667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.087398 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.089424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.198919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1342/2680] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:30 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:30.202987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.214386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef34c3-4c60-43f0-996f-606280ddc179 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.218999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.219708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.220018 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.220302 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.222131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.223797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.234847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1339/2681] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:30.240722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.251010 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798f9a8-6d60-4b27-b0e9-07c4c93c92b8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.256013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.256929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.257214 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.257488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] PATH_INFO: `/v3/projects/111e6b8938de4f878ddfd27d419f4557/users/d510fb5c200d48d58cd3323d08aae1bc/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.272800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111e6b8938de4f878ddfd27d419f4557, user_id=d510fb5c200d48d58cd3323d08aae1bc, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.275394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.368979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1343/2682] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:30 2022] PUT /identity/v3/projects/111e6b8938de4f878ddfd27d419f4557/users/d510fb5c200d48d58cd3323d08aae1bc/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:30.373035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.383371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd4c5b-cb94-479e-9925-7d2cd256a866 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.404584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:30.440034 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b6cca-4112-4bac-beb7-9bd69c11285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.440752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b6cca-4112-4bac-beb7-9bd69c11285d admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.441042 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b6cca-4112-4bac-beb7-9bd69c11285d admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.441315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b6cca-4112-4bac-beb7-9bd69c11285d admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.442063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b6cca-4112-4bac-beb7-9bd69c11285d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.443701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b6cca-4112-4bac-beb7-9bd69c11285d admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.453880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1340/2683] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:30.458651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.464070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:30.469379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f138-8887-4632-bf65-05ddfd98e611 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.474683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.475571 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.475857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.476134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] PATH_INFO: `/v3/projects/111e6b8938de4f878ddfd27d419f4557/users/d510fb5c200d48d58cd3323d08aae1bc/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.478170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111e6b8938de4f878ddfd27d419f4557, user_id=d510fb5c200d48d58cd3323d08aae1bc, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.480842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.491316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1344/2684] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:30 2022] PUT /identity/v3/projects/111e6b8938de4f878ddfd27d419f4557/users/d510fb5c200d48d58cd3323d08aae1bc/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:30.504739 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.505033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.505704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.533177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] MFA Rules not processed for user `d510fb5c200d48d58cd3323d08aae1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:30.534407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:30.604574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1341/2685] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:30.616205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.617950 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:30.674925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e758-15ce-4458-921f-0d79e5bee0f5 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.678886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f9987-b16c-4478-926e-31cbd5f64eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.681868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f9987-b16c-4478-926e-31cbd5f64eaf service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.681868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f9987-b16c-4478-926e-31cbd5f64eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.681868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f9987-b16c-4478-926e-31cbd5f64eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.689922 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f9987-b16c-4478-926e-31cbd5f64eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.691851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f9987-b16c-4478-926e-31cbd5f64eaf service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:30.701407 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1345/2686] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:30.767752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:30.830729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4ace-bcc1-4cdf-8862-a75eab8d9a68 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:30.835406 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:30.836273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:30.836588 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:30.836869 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] PATH_INFO: `/v3/users/705403ae0e644b49858f2e86e00014f6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:30.838090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=705403ae0e644b49858f2e86e00014f6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:30.840031 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:31.069636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '705403ae0e644b49858f2e86e00014f6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:31.070224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '705403ae0e644b49858f2e86e00014f6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:31.071469 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:31.077670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '705403ae0e644b49858f2e86e00014f6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:31.078356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '705403ae0e644b49858f2e86e00014f6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:31.161726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '705403ae0e644b49858f2e86e00014f6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:31.179829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1342/2687] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:02:30 2022] DELETE /identity/v3/users/705403ae0e644b49858f2e86e00014f6 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:31.184476 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:31.184995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:31.185108 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:31.203481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:31.204703 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:31.264486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1346/2688] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:31.274618 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:31.276135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:31.333492 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c56d88-98c3-402d-9b07-d9b50e4ab40c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:31.336972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:31.337614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:31.337897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:31.338173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:31.347479 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:31.348850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:31.358286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1343/2689] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:31.880477 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:31.882212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:31.894670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899afd21-db01-450f-a790-1e2202c7ebf5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:31.905035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:31.905035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:31.905035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:31.905035 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] PATH_INFO: `/v3/projects/c737a4c54dc94f4196be29e3e6a82bf5` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:31.906290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c737a4c54dc94f4196be29e3e6a82bf5)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:31.908362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:32.028730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c737a4c54dc94f4196be29e3e6a82bf5', 'request_id': 'req-9e37ca51-752c-49a8-b142-5791d609c170'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.111938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c737a4c54dc94f4196be29e3e6a82bf5', 'request_id': 'req-9e37ca51-752c-49a8-b142-5791d609c170'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.119901 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c737a4c54dc94f4196be29e3e6a82bf5', 'request_id': 'req-9e37ca51-752c-49a8-b142-5791d609c170'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.131842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] The token cache is being invalidate because project c737a4c54dc94f4196be29e3e6a82bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:02:32.132051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.132733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.134291 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1347/2690] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:02:31 2022] DELETE /identity/v3/projects/c737a4c54dc94f4196be29e3e6a82bf5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:32.138228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:32.149194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef1f84-c753-48bd-aea4-d96f700854e1 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:32.210059 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:32.210945 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:32.211231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:32.211508 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] PATH_INFO: `/v3/users/d510fb5c200d48d58cd3323d08aae1bc` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:32.212752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d510fb5c200d48d58cd3323d08aae1bc)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:32.214806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:32.401005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd510fb5c200d48d58cd3323d08aae1bc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.401595 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd510fb5c200d48d58cd3323d08aae1bc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.408837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd510fb5c200d48d58cd3323d08aae1bc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.411191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd510fb5c200d48d58cd3323d08aae1bc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.483575 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-af5f2a6c-15c8-4092-b595-906aee53fab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd510fb5c200d48d58cd3323d08aae1bc'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:32.490128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:32.502750 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1344/2691] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:02:32 2022] DELETE /identity/v3/users/d510fb5c200d48d58cd3323d08aae1bc => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:32.924013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:32.926846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:32.943162 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37ca51-752c-49a8-b142-5791d609c170 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:32.957100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:33.013424 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.014296 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.014577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.014871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] PATH_INFO: `/v3/projects/111e6b8938de4f878ddfd27d419f4557` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.016073 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=111e6b8938de4f878ddfd27d419f4557)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.018016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.156323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111e6b8938de4f878ddfd27d419f4557', 'request_id': 'req-19136292-ed9e-423f-b766-e33b195fc5c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:33.231859 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111e6b8938de4f878ddfd27d419f4557', 'request_id': 'req-19136292-ed9e-423f-b766-e33b195fc5c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:33.240238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111e6b8938de4f878ddfd27d419f4557', 'request_id': 'req-19136292-ed9e-423f-b766-e33b195fc5c5'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:33.250803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] The token cache is being invalidate because project 111e6b8938de4f878ddfd27d419f4557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:02:33.251096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:33.251805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-19136292-ed9e-423f-b766-e33b195fc5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:33.253896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1348/2692] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:02:32 2022] DELETE /identity/v3/projects/111e6b8938de4f878ddfd27d419f4557 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:02:33.258833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.259315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.259481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.277680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:33.278883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:33.340194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1345/2693] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:02:33.348523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.348836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.349159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.368060 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:33.369499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:33.377429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1349/2694] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:33.381909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:33.391820 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250adbdb-0e8f-4611-8957-af3425dcfd11 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:33.398066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.398722 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.399005 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.399280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.400017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:02:33.400328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.401996 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.414976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1346/2695] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:02:33.419365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:33.429958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad280a-808e-4c35-9e66-f7df22966c0b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:33.434670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.435322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.435606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.435882 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.436626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.438744 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.559794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1350/2696] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:33 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:33.564008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:33.575355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb074-d849-4b0c-9e64-90dfc0d06385 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:33.632612 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.633286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.633576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.633854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.634552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.636481 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.730514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1347/2697] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:33 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:33.734426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:33.744805 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12937-be38-4ea2-a822-56ab796d1a62 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:33.749633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.750310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.750603 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.750889 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.751628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.753809 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.764275 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1351/2698] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:33.768427 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:33.779074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a813a-84b1-4557-8d90-d35da239a844 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:33.783718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91320198-4110-4a23-973a-a3f5814ed779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.784405 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320198-4110-4a23-973a-a3f5814ed779 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.784795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320198-4110-4a23-973a-a3f5814ed779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.785027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91320198-4110-4a23-973a-a3f5814ed779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.785707 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91320198-4110-4a23-973a-a3f5814ed779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.787303 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91320198-4110-4a23-973a-a3f5814ed779 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.797068 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1348/2699] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:33.801195 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:33.813835 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8d98-1825-4d4d-8e85-e2cce22bf689 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:33.816569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:33.817452 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.817741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.818023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] PATH_INFO: `/v3/projects/55fc7215647f40d2a11b55374bd0d03b/users/d63e6e33a44746e294ce8e32eda70ee6/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.833747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fc7215647f40d2a11b55374bd0d03b, user_id=d63e6e33a44746e294ce8e32eda70ee6, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:33.836709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:33.941671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1352/2700] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:33 2022] PUT /identity/v3/projects/55fc7215647f40d2a11b55374bd0d03b/users/d63e6e33a44746e294ce8e32eda70ee6/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:33.953510 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:33.953799 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:33.954078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:33.981693 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] MFA Rules not processed for user `d63e6e33a44746e294ce8e32eda70ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:33.982948 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:34.025106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:34.047336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1349/2701] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:34.057904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:34.059619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:34.105891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:34.117623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7109eb-3a64-4399-ab9b-6c473a5dd604 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:34.121310 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c4dbf-0507-4601-a38c-a8034a185a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:34.121958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c4dbf-0507-4601-a38c-a8034a185a7c service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:34.122248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c4dbf-0507-4601-a38c-a8034a185a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:34.122530 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c4dbf-0507-4601-a38c-a8034a185a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:34.131565 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c4dbf-0507-4601-a38c-a8034a185a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:34.133157 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c4dbf-0507-4601-a38c-a8034a185a7c service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:34.142633 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1353/2702] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:43.883888 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:43.945838 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb74e2-019d-4d0d-8ded-917be031b0d7 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:43.950647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:43.951534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:43.951816 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:43.952091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] PATH_INFO: `/v3/users/d63e6e33a44746e294ce8e32eda70ee6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:43.953316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63e6e33a44746e294ce8e32eda70ee6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:43.955252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:44.129256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63e6e33a44746e294ce8e32eda70ee6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:44.129922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63e6e33a44746e294ce8e32eda70ee6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:44.137351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63e6e33a44746e294ce8e32eda70ee6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:44.137972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63e6e33a44746e294ce8e32eda70ee6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:44.197498 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=7 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:44.206954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63e6e33a44746e294ce8e32eda70ee6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:44.226777 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1350/2703] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:02:43 2022] DELETE /identity/v3/users/d63e6e33a44746e294ce8e32eda70ee6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:44.235907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:44.236229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:44.236523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:44.254778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:44.256274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:44.319993 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1354/2704] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:44.329763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:44.331259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:44.387549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cf0e7-b3dd-4289-ab6b-63e90f243f2c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:44.391124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b9312-811e-4ecd-b611-6c4c4717f843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:44.391768 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9312-811e-4ecd-b611-6c4c4717f843 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:44.392051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9312-811e-4ecd-b611-6c4c4717f843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:44.392325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9312-811e-4ecd-b611-6c4c4717f843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:44.401496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b9312-811e-4ecd-b611-6c4c4717f843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:44.403027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b9312-811e-4ecd-b611-6c4c4717f843 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:44.413147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1351/2705] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:44.810792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:44.812280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:44.826134 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a26a91-5f6d-4b99-bd6b-12ca620456c5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:44.831036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:44.831938 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:44.832339 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:44.832800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] PATH_INFO: `/v3/projects/55fc7215647f40d2a11b55374bd0d03b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:44.834379 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55fc7215647f40d2a11b55374bd0d03b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:44.836431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:44.954197 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fc7215647f40d2a11b55374bd0d03b', 'request_id': 'req-e769c397-075b-4b15-9e68-b3f0a5997a6e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:45.031940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fc7215647f40d2a11b55374bd0d03b', 'request_id': 'req-e769c397-075b-4b15-9e68-b3f0a5997a6e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:45.039367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fc7215647f40d2a11b55374bd0d03b', 'request_id': 'req-e769c397-075b-4b15-9e68-b3f0a5997a6e'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:45.049752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] The token cache is being invalidate because project 55fc7215647f40d2a11b55374bd0d03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:02:45.050043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:45.050677 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-e769c397-075b-4b15-9e68-b3f0a5997a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:45.052816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1355/2706] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:02:44 2022] DELETE /identity/v3/projects/55fc7215647f40d2a11b55374bd0d03b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:45.057644 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.058104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.058218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.065884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:45.089170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:45.090451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:45.160180 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1352/2707] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:45.167460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.167757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.168044 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.187468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:45.188979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:45.197991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1356/2708] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:45.202263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.212734 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f47d7-0cd2-4a0d-b63a-8e07624db8e3 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.217202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.217850 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.218134 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.218415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.219135 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:02:45.219511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.221116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.233030 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1353/2709] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:02:45.237322 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.247720 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c929c-5662-4a86-a34f-c86501bd278a None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.253707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.254869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.255158 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.255706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.256198 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.258222 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.341781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1357/2710] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:45 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:02:45.346015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.356087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8de0a-8984-49f7-bdc9-805e726f69dd admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.411935 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.412627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.412912 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.413215 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.413909 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.415812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.502578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1354/2711] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:45 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:45.507885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.518619 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12ca30-86f2-47c6-a857-63a5f7d88d57 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.523700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.524358 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.524699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.524984 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.525711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.527317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.539037 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1358/2712] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:45.543125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.554310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c93b2f-ad0d-4283-9bd9-8490ba58bfc9 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.558871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44199d5-75ac-40b8-b53e-15771eda2fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.559515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44199d5-75ac-40b8-b53e-15771eda2fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.559795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44199d5-75ac-40b8-b53e-15771eda2fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.560066 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44199d5-75ac-40b8-b53e-15771eda2fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.560837 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44199d5-75ac-40b8-b53e-15771eda2fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.562448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44199d5-75ac-40b8-b53e-15771eda2fe9 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.573518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1355/2713] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:45.577660 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.587620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2a69b-88b9-4654-bc10-46adaa6769dd admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.592260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.593165 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.593453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.593728 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] PATH_INFO: `/v3/projects/301f6e69069c487f93db3527d4a5624d/users/6c28b50dd4f24b1eae7fe529ca1ba988/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.608701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=301f6e69069c487f93db3527d4a5624d, user_id=6c28b50dd4f24b1eae7fe529ca1ba988, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.611333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.678976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1359/2714] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:45 2022] PUT /identity/v3/projects/301f6e69069c487f93db3527d4a5624d/users/6c28b50dd4f24b1eae7fe529ca1ba988/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:45.690793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.691096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.691382 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.718312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] MFA Rules not processed for user `6c28b50dd4f24b1eae7fe529ca1ba988`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:45.719549 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:45.780274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:45.783800 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1356/2715] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:45.799457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:45.801362 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:45.823816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:45.858668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97e79-e075-445b-9e22-11c2b5cb1c2b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:45.862397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066b6b-cf40-4153-8560-5e558e88d64c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:45.863049 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066b6b-cf40-4153-8560-5e558e88d64c service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:45.863334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066b6b-cf40-4153-8560-5e558e88d64c service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:45.863662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066b6b-cf40-4153-8560-5e558e88d64c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:45.872211 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066b6b-cf40-4153-8560-5e558e88d64c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:45.873693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066b6b-cf40-4153-8560-5e558e88d64c service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:45.885628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1360/2716] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:51.101219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:51.169806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef3200-bffe-4f4e-8d31-a2bc075ffe6f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:51.174325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:51.175192 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:51.175471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:51.175764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] PATH_INFO: `/v3/users/6c28b50dd4f24b1eae7fe529ca1ba988` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:51.177652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c28b50dd4f24b1eae7fe529ca1ba988)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:51.179574 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:51.352240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c28b50dd4f24b1eae7fe529ca1ba988'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:51.352920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c28b50dd4f24b1eae7fe529ca1ba988'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:51.359919 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c28b50dd4f24b1eae7fe529ca1ba988'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:51.360586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c28b50dd4f24b1eae7fe529ca1ba988'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:51.420142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c28b50dd4f24b1eae7fe529ca1ba988'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:51.439628 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1357/2717] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:02:51 2022] DELETE /identity/v3/users/6c28b50dd4f24b1eae7fe529ca1ba988 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:51.444320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:51.444872 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:51.444972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:51.462680 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:51.463887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:51.522907 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1361/2718] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:51.533072 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:51.534496 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:51.595538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e32e5-8c29-4e8f-a293-89126e60a84a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:51.599745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375c747-bf4b-43b5-8c6b-2e08d7ce4126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:51.600386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375c747-bf4b-43b5-8c6b-2e08d7ce4126 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:51.600703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375c747-bf4b-43b5-8c6b-2e08d7ce4126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:51.600977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375c747-bf4b-43b5-8c6b-2e08d7ce4126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:51.609886 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375c747-bf4b-43b5-8c6b-2e08d7ce4126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:51.611326 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375c747-bf4b-43b5-8c6b-2e08d7ce4126 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:51.620790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1358/2719] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:52.124705 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.126132 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:52.139156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed2c8c-a731-4abd-9db3-856d9f7a6c7b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.143790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.144696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.150383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.150383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] PATH_INFO: `/v3/projects/301f6e69069c487f93db3527d4a5624d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.150383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=301f6e69069c487f93db3527d4a5624d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.150383 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:52.266431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301f6e69069c487f93db3527d4a5624d', 'request_id': 'req-bcf432b9-89f7-4bd8-bddf-56680c424d59'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:52.338219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301f6e69069c487f93db3527d4a5624d', 'request_id': 'req-bcf432b9-89f7-4bd8-bddf-56680c424d59'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:52.344977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301f6e69069c487f93db3527d4a5624d', 'request_id': 'req-bcf432b9-89f7-4bd8-bddf-56680c424d59'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:52.356465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] The token cache is being invalidate because project 301f6e69069c487f93db3527d4a5624d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:02:52.356785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:52.357426 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-bcf432b9-89f7-4bd8-bddf-56680c424d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:02:52.358996 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1362/2720] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:02:52 2022] DELETE /identity/v3/projects/301f6e69069c487f93db3527d4a5624d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:52.364116 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.365221 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.365396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.385697 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:52.386814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:52.403665 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:52.446459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1359/2721] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:52.454200 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.454688 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.454817 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.472928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:52.474663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:52.483349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1363/2722] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:52.487449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.499085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae61f9d-e90f-4f73-9350-839803f9dd22 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.503907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.504602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.504884 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.505472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.505965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:02:52.506236 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.507821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:52.523187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1360/2723] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 04:02:52.524577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.534287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344e582-6891-4a2f-88a2-e984763cf3e7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.539908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.540595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.540884 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.541164 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.541868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.543783 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:52.651184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1364/2724] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:52 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:52.655371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.666396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d9911-a65e-482d-8569-61a3cf84bfb0 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.722262 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.722928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.723213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.723488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.724742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.726755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:52.797616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1361/2725] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:52 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:02:52.801955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.813620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263cf9f4-e7bd-4d02-9d70-3c2ae0449518 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.859405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.859405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.859405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.859405 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.859851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.859851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:52.859851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1365/2726] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:52.862337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.872895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b79e3-4834-4391-8cbb-2e808cf0e8fe admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.877623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d94a1f-694c-4c91-b2b6-33fab0406add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.878290 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d94a1f-694c-4c91-b2b6-33fab0406add admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.878577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d94a1f-694c-4c91-b2b6-33fab0406add admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.878903 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d94a1f-694c-4c91-b2b6-33fab0406add admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.879636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d94a1f-694c-4c91-b2b6-33fab0406add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.881364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d94a1f-694c-4c91-b2b6-33fab0406add admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:52.891682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1362/2727] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:52.896576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:52.907004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7d5b2-30c2-4e89-903d-ab8183964d24 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:52.912317 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:52.913392 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:52.913681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:52.913962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] PATH_INFO: `/v3/projects/ffc0a8e4ff234b95bbf510878bfae842/users/3910d7ee5c5e4b8bb1b35639eec8fa67/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:52.916212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:52.929374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc0a8e4ff234b95bbf510878bfae842, user_id=3910d7ee5c5e4b8bb1b35639eec8fa67, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:52.932020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:53.014981 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1366/2728] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:52 2022] PUT /identity/v3/projects/ffc0a8e4ff234b95bbf510878bfae842/users/3910d7ee5c5e4b8bb1b35639eec8fa67/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:53.034495 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.034783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.035061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.062833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] MFA Rules not processed for user `3910d7ee5c5e4b8bb1b35639eec8fa67`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:53.064027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:53.129088 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1363/2729] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:53.133940 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:53.196878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5d53-5796-473b-812d-92382e248cc5 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:53.201590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:53.202258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.202550 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.202848 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.203556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:53.205516 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:53.325638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1367/2730] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:53 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:53.330006 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:53.342283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8c14c-07c6-45bb-a4aa-f7ad39fcb03b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:53.354634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:53.399709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:53.400396 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.400675 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.400952 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.401717 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:53.403626 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:53.471170 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1364/2731] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:53 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:53.475144 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:53.485695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377920b-0535-45ce-bbcc-3a415a6d82f3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:53.490496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:53.491221 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.491506 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.491784 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.492564 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:53.494143 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:53.505139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1368/2732] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:53.509714 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:53.519807 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f75e42-3965-4c5b-a5fa-f179bd8a9f34 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:53.525466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739b4b3-4945-4089-a5e3-9befe93bb842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:53.526164 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739b4b3-4945-4089-a5e3-9befe93bb842 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.526452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739b4b3-4945-4089-a5e3-9befe93bb842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.526736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739b4b3-4945-4089-a5e3-9befe93bb842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.527457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739b4b3-4945-4089-a5e3-9befe93bb842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:53.529106 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739b4b3-4945-4089-a5e3-9befe93bb842 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:53.539982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1365/2733] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:53.544139 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:53.554463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4b2ef-79e6-429d-a1e2-f9926d596ec3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:53.559365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:53.560247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.560558 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.560838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] PATH_INFO: `/v3/projects/fcbf758a9fab48909470e9ba2fa5f9a2/users/7363d81215e04a87b250fe6dd1bf3742/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.582004 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbf758a9fab48909470e9ba2fa5f9a2, user_id=7363d81215e04a87b250fe6dd1bf3742, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:53.585375 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:53.666674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1369/2734] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:53 2022] PUT /identity/v3/projects/fcbf758a9fab48909470e9ba2fa5f9a2/users/7363d81215e04a87b250fe6dd1bf3742/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:53.730937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.731167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.731410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.758978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] MFA Rules not processed for user `7363d81215e04a87b250fe6dd1bf3742`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:53.760184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:53.824702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1366/2735] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:53.829316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:53.894641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c47bc-8b87-4701-9c00-48fcb9429d80 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:53.899673 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:53.900343 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:53.900694 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:53.900978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:53.901693 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:53.903628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.004670 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1370/2736] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:02:53 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:02:54.009151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:54.019916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36928-9867-4fd6-bc62-1c911bda396e None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:54.075979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:54.076679 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.076960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.077231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.077921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:54.079981 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.169894 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1367/2737] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:02:54 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:02:54.173963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:54.185242 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbbad4-38b7-4cc7-924f-128e8afa8ef7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:54.190203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:54.190888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.191237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.191457 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.192243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:54.194122 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.205970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1371/2738] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:54.210416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:54.220954 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10404a7-0a3e-42e4-b347-91e75c00ea93 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:54.226452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:54.233259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.233259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.233259 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] PATH_INFO: `/v3/projects/5cc524560b5546b1a1cb6c6a105935db/users/a5f17363f81f48338aa921f2e6f0494d/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.241350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:54.244450 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc524560b5546b1a1cb6c6a105935db, user_id=a5f17363f81f48338aa921f2e6f0494d, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:54.247196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.306204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1368/2739] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:54 2022] PUT /identity/v3/projects/5cc524560b5546b1a1cb6c6a105935db/users/a5f17363f81f48338aa921f2e6f0494d/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:54.310065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:54.321531 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0958564-b074-43ad-869f-aac83f54e23b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:54.325380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:02:54.377869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b2a16-edb2-4bea-b02b-5f4c725451bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:54.378529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b2a16-edb2-4bea-b02b-5f4c725451bb admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.378813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b2a16-edb2-4bea-b02b-5f4c725451bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.379084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b2a16-edb2-4bea-b02b-5f4c725451bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.379796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b2a16-edb2-4bea-b02b-5f4c725451bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:54.381643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b2a16-edb2-4bea-b02b-5f4c725451bb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.392659 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1372/2740] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:02:54.397369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:54.408970 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abec29a-9d43-4be6-bfa4-0e2349ba4c0a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:54.414499 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:54.415386 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.415674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.415951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] PATH_INFO: `/v3/projects/5cc524560b5546b1a1cb6c6a105935db/users/a5f17363f81f48338aa921f2e6f0494d/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.417950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc524560b5546b1a1cb6c6a105935db, user_id=a5f17363f81f48338aa921f2e6f0494d, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:54.420603 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.431817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1369/2741] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:02:54 2022] PUT /identity/v3/projects/5cc524560b5546b1a1cb6c6a105935db/users/a5f17363f81f48338aa921f2e6f0494d/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:02:54.442536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.442826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.443124 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.471585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] MFA Rules not processed for user `a5f17363f81f48338aa921f2e6f0494d`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:54.472838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:54.542671 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1373/2742] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:02:54.554457 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:54.556282 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:54.621466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9308cfe-41da-4635-84d6-e929073712d5 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:54.625183 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e3542-7ad5-49fc-8368-95f652a88a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:54.625938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e3542-7ad5-49fc-8368-95f652a88a29 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:54.640113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e3542-7ad5-49fc-8368-95f652a88a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:54.640113 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e3542-7ad5-49fc-8368-95f652a88a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:54.689448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e3542-7ad5-49fc-8368-95f652a88a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:54.690893 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e3542-7ad5-49fc-8368-95f652a88a29 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:54.701187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1370/2743] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:56.719233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:56.720655 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:56.732528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd140a6-2b85-49f7-8717-b33b2a803551 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:56.736151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:56.736844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:56.737130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:56.737403 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:56.794374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:56.795832 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:56.806264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1374/2744] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:02:58.234616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:58.241331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:58.241331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:58.253319 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] MFA Rules not processed for user `18d67c48b3a5445296a2069be87ebae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:02:58.254534 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:02:58.315591 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1371/2745] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:02:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:02:58.360394 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:02:58.370937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5a047-1156-4c4b-9f62-c9a096e57e0b service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:02:58.379823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed575f-3ab6-48c8-b3cf-7c21f3721e55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:02:58.379823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed575f-3ab6-48c8-b3cf-7c21f3721e55 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:02:58.379823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed575f-3ab6-48c8-b3cf-7c21f3721e55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:02:58.379823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed575f-3ab6-48c8-b3cf-7c21f3721e55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:02:58.387580 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed575f-3ab6-48c8-b3cf-7c21f3721e55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:02:58.389142 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed575f-3ab6-48c8-b3cf-7c21f3721e55 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:02:58.399443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1375/2746] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:02:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:03:00.447684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:00.450634 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:00.523878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:00.588337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14799c-f304-46d8-8a8f-712714ca0c3f None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:00.593547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:00.594224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:00.594493 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:00.594767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:00.598636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:00.611335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:00.613002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:00.624582 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1372/2747] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:03:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:03:02.340992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd89e-5564-48e1-9d07-faf5bac66bda None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:02.340992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd89e-5564-48e1-9d07-faf5bac66bda None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:02.340992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdd89e-5564-48e1-9d07-faf5bac66bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:02.359669 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d3bdd89e-5564-48e1-9d07-faf5bac66bda None None] MFA Rules not processed for user `f1d4360de70b4beea3456aa3dc85d5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:03:02.360974 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d3bdd89e-5564-48e1-9d07-faf5bac66bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:02.370016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=100 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:02.419218 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1376/2748] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:03:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:03:03.322087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:03.389991 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c72363-9989-4f7c-bd14-86a3b29c05a8 service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:03.394959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:03.396782 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:03.397048 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:03.397331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] PATH_INFO: `/v3/users/3910d7ee5c5e4b8bb1b35639eec8fa67` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:03.398613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3910d7ee5c5e4b8bb1b35639eec8fa67)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:03.400712 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:03.594177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3910d7ee5c5e4b8bb1b35639eec8fa67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:03.595416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3910d7ee5c5e4b8bb1b35639eec8fa67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:03.606198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3910d7ee5c5e4b8bb1b35639eec8fa67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:03.608001 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3910d7ee5c5e4b8bb1b35639eec8fa67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:03.677767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3910d7ee5c5e4b8bb1b35639eec8fa67'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:03.697202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1373/2749] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:03:03 2022] DELETE /identity/v3/users/3910d7ee5c5e4b8bb1b35639eec8fa67 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:03.701891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:03.705280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:03.705280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:03.720811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:03:03.722033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:03.799356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1377/2750] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:03:03.816211 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:03.818003 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:03.855808 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:03.908976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181eccbd-325c-43da-a5a3-c32b87a54e1f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:03.908976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:03.908976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:03.908976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:03.909357 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:03.919081 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:03.920623 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:03.931417 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1374/2751] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:03:04.561234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:04.562891 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:04.576640 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd656c9-f5ef-417d-b8aa-edc287dfbff1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:04.583370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:04.584282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:04.584597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:04.584896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] PATH_INFO: `/v3/projects/ffc0a8e4ff234b95bbf510878bfae842` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:04.586274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc0a8e4ff234b95bbf510878bfae842)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:04.588454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:04.721667 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc0a8e4ff234b95bbf510878bfae842', 'request_id': 'req-ed011d6c-a19f-4c17-af93-293d62f76983'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:04.825886 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc0a8e4ff234b95bbf510878bfae842', 'request_id': 'req-ed011d6c-a19f-4c17-af93-293d62f76983'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:04.834172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc0a8e4ff234b95bbf510878bfae842', 'request_id': 'req-ed011d6c-a19f-4c17-af93-293d62f76983'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:04.845437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] The token cache is being invalidate because project ffc0a8e4ff234b95bbf510878bfae842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:03:04.845732 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:04.846447 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:04.848069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1378/2752] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:03:04 2022] DELETE /identity/v3/projects/ffc0a8e4ff234b95bbf510878bfae842 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:03:04.852790 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:04.863356 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b36475-d7b3-4b9e-ab55-41d33f4522f4 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:04.938240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:04.938240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:04.938240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:04.938240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] PATH_INFO: `/v3/users/7363d81215e04a87b250fe6dd1bf3742` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:04.938729 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7363d81215e04a87b250fe6dd1bf3742)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:04.941276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:05.114408 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7363d81215e04a87b250fe6dd1bf3742'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.115002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7363d81215e04a87b250fe6dd1bf3742'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.122154 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7363d81215e04a87b250fe6dd1bf3742'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.122797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7363d81215e04a87b250fe6dd1bf3742'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.219929 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7363d81215e04a87b250fe6dd1bf3742'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.238257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1375/2753] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:03:04 2022] DELETE /identity/v3/users/7363d81215e04a87b250fe6dd1bf3742 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:05.685280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:05.686920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:05.703869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed011d6c-a19f-4c17-af93-293d62f76983 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:05.746219 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:05.764118 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:05.765024 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:05.765312 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:05.765592 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] PATH_INFO: `/v3/projects/fcbf758a9fab48909470e9ba2fa5f9a2` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:05.766777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbf758a9fab48909470e9ba2fa5f9a2)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:05.768740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:05.871740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbf758a9fab48909470e9ba2fa5f9a2', 'request_id': 'req-3d2260bc-e99d-47de-90be-aaafe7a3c001'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.960364 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbf758a9fab48909470e9ba2fa5f9a2', 'request_id': 'req-3d2260bc-e99d-47de-90be-aaafe7a3c001'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.968408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbf758a9fab48909470e9ba2fa5f9a2', 'request_id': 'req-3d2260bc-e99d-47de-90be-aaafe7a3c001'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.980349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] The token cache is being invalidate because project fcbf758a9fab48909470e9ba2fa5f9a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:03:05.980701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.981381 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:05.982960 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1379/2754] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:03:05 2022] DELETE /identity/v3/projects/fcbf758a9fab48909470e9ba2fa5f9a2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:05.987017 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:05.988744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:06.002458 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9218c-97f2-4359-b891-4d5e8b4384b1 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:06.048468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:06.060984 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:06.061880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:06.062422 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:06.062700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] PATH_INFO: `/v3/users/a5f17363f81f48338aa921f2e6f0494d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:06.063917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f17363f81f48338aa921f2e6f0494d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:06.065904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:06.309304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f17363f81f48338aa921f2e6f0494d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:06.309964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f17363f81f48338aa921f2e6f0494d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:06.317924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f17363f81f48338aa921f2e6f0494d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:06.318767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f17363f81f48338aa921f2e6f0494d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:06.390878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-ecc13fe1-f7af-4178-9e67-c19084de733a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f17363f81f48338aa921f2e6f0494d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:06.409944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1376/2755] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:03:05 2022] DELETE /identity/v3/users/a5f17363f81f48338aa921f2e6f0494d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:03:06.844540 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:06.846020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:06.860449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2260bc-e99d-47de-90be-aaafe7a3c001 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:06.924910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:06.925777 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:06.926061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:06.926346 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] PATH_INFO: `/v3/projects/5cc524560b5546b1a1cb6c6a105935db` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:06.928050 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc524560b5546b1a1cb6c6a105935db)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:06.930020 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.102097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc524560b5546b1a1cb6c6a105935db', 'request_id': 'req-719bf3b1-a53b-4605-afee-5cc84ec6e394'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:07.214388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc524560b5546b1a1cb6c6a105935db', 'request_id': 'req-719bf3b1-a53b-4605-afee-5cc84ec6e394'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:07.224717 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc524560b5546b1a1cb6c6a105935db', 'request_id': 'req-719bf3b1-a53b-4605-afee-5cc84ec6e394'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:07.234615 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] The token cache is being invalidate because project 5cc524560b5546b1a1cb6c6a105935db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:03:07.234995 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:07.235841 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-719bf3b1-a53b-4605-afee-5cc84ec6e394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:07.237708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1380/2756] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:03:06 2022] DELETE /identity/v3/projects/5cc524560b5546b1a1cb6c6a105935db => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:07.243342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.243868 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.243962 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.263719 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:03:07.265121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:07.327368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1377/2757] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:03:07.335776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.336063 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.336413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.342700 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:07.355523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:03:07.356696 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:07.364752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1381/2758] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:03:07.368987 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:07.379074 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3c83-f710-487f-8b9d-375cef1c86ae None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:07.383784 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:07.384442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.384757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.385033 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.385760 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:03:07.386069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:07.387670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.405794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1378/2759] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:03:07.406881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:07.421570 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d87ba-6f0f-4f81-a460-ffbb4812d1a1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:07.426796 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:07.427461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.427764 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.428434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.428895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:07.430814 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.537337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1382/2760] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:03:07 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:03:07.538602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:07.549938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606517d-b7c4-49ce-a3c8-f5db615724ce admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:07.601971 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:07.608853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:07.609471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.609757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.610038 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.610767 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:07.612721 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.703040 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1379/2761] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:03:07 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:03:07.707170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:07.718578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1714-d069-462c-86a0-dc575817c2f8 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:07.723509 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:07.724213 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.724526 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.724813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.725536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:07.727203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.739120 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1383/2762] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:03:07.743037 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:07.753485 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a8d58-8d3b-46bb-a3b7-b587c70f5bb6 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:07.758874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861cbd8-8e08-4619-ae51-53813c7becc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:07.759822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861cbd8-8e08-4619-ae51-53813c7becc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.759822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861cbd8-8e08-4619-ae51-53813c7becc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.760140 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861cbd8-8e08-4619-ae51-53813c7becc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.760847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861cbd8-8e08-4619-ae51-53813c7becc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:07.762489 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861cbd8-8e08-4619-ae51-53813c7becc8 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.773415 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1380/2763] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:03:07.777851 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:07.788898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f9030-2074-445f-90af-bfed5aac2300 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:07.793743 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:07.794629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.794913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.795253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] PATH_INFO: `/v3/projects/22ed1117e7544bfabbe4153642ed5f1e/users/385c45cba17743fcb1bc52126ad46a84/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.810707 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ed1117e7544bfabbe4153642ed5f1e, user_id=385c45cba17743fcb1bc52126ad46a84, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:07.813714 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:07.911869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1384/2764] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:03:07 2022] PUT /identity/v3/projects/22ed1117e7544bfabbe4153642ed5f1e/users/385c45cba17743fcb1bc52126ad46a84/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:07.922138 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:07.922436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:07.922711 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:07.953431 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] MFA Rules not processed for user `385c45cba17743fcb1bc52126ad46a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:03:07.954651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:08.023972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1381/2765] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:03:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:03:08.037811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:08.039649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:08.105333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50738a6f-cdf9-4746-bed8-b1c4b9bfc957 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:08.108988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0839d0d-3f68-4797-9aae-dcc715229ee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:08.109638 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0839d0d-3f68-4797-9aae-dcc715229ee3 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:08.109921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0839d0d-3f68-4797-9aae-dcc715229ee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:08.110191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0839d0d-3f68-4797-9aae-dcc715229ee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:08.119458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0839d0d-3f68-4797-9aae-dcc715229ee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:08.120967 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0839d0d-3f68-4797-9aae-dcc715229ee3 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:08.131058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1385/2766] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:03:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:03:18.020300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:18.125898 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:20.044473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:20.109314 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87c647-c3cf-44a0-8558-34bebe40c234 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:20.113998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:20.114809 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:20.115090 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:20.115365 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] PATH_INFO: `/v3/users/385c45cba17743fcb1bc52126ad46a84` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:20.116593 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=385c45cba17743fcb1bc52126ad46a84)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:20.118527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:20.313324 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '385c45cba17743fcb1bc52126ad46a84'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:20.313990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '385c45cba17743fcb1bc52126ad46a84'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:20.321515 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '385c45cba17743fcb1bc52126ad46a84'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:20.322179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '385c45cba17743fcb1bc52126ad46a84'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:20.384833 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '385c45cba17743fcb1bc52126ad46a84'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:20.404854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1382/2767] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:03:20 2022] DELETE /identity/v3/users/385c45cba17743fcb1bc52126ad46a84 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:20.409487 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:20.409971 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:20.410084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:20.431131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:03:20.432489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:20.496041 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1386/2768] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:03:20.520737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:20.520737 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:20.571084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d730a03-be22-40e3-8d40-34f6c938df51 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:20.574862 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:20.575550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:20.575836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:20.576133 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:20.586554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:20.588091 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:20.599167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1383/2769] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:03:21.021629 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:03:21.023111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:03:21.035439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86346d-97cd-48eb-b49c-8ab3b36a3337 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:03:21.040148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:03:21.041059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:03:21.041481 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:03:21.041767 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] PATH_INFO: `/v3/projects/22ed1117e7544bfabbe4153642ed5f1e` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:03:21.043021 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ed1117e7544bfabbe4153642ed5f1e)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:03:21.044985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:03:21.147811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ed1117e7544bfabbe4153642ed5f1e', 'request_id': 'req-93292127-555b-4aa5-8904-730d26e218c7'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:21.205801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ed1117e7544bfabbe4153642ed5f1e', 'request_id': 'req-93292127-555b-4aa5-8904-730d26e218c7'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:21.213578 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ed1117e7544bfabbe4153642ed5f1e', 'request_id': 'req-93292127-555b-4aa5-8904-730d26e218c7'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:21.224813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] The token cache is being invalidate because project 22ed1117e7544bfabbe4153642ed5f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:03:21.225106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:21.225757 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:03:21.227429 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1387/2770] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:03:21 2022] DELETE /identity/v3/projects/22ed1117e7544bfabbe4153642ed5f1e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:03:30.593537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:03:31.221048 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:04:30.974198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:04:30.976671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:04:31.036715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18744e-05d9-48f8-97ec-dc685737f04f service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:04:31.090725 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:04:31.091452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:04:31.091672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:04:31.091950 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:04:31.093795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:04:31.190747 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:04:31.192672 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85790204-e7f5-4294-b51b-4f5de03dba5a service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:04:31.215764 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1384/2771] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:04:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:04:32.834511 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:04:32.836046 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:04:32.895910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93292127-555b-4aa5-8904-730d26e218c7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:04:32.899467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:04:32.900115 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:04:32.900404 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:04:32.900721 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:04:32.902788 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:04:32.916068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:04:32.918299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:04:32.929276 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1388/2772] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:04:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:04:41.212414 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:04:42.924345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=26 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:04:48.597748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:04:48.598331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:04:48.598331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:04:48.616444 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] MFA Rules not processed for user `18d67c48b3a5445296a2069be87ebae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:04:48.617692 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:04:48.675576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1385/2773] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:04:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:04:52.204978 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:04:52.214351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239f016-5cda-4569-b889-f526402d8e6f service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:04:52.218510 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:04:52.219254 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:04:52.219663 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:04:52.220019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:04:52.221826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:04:52.232946 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:04:52.234544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6ca7f-3029-44c8-acef-5e9cc3374ebd service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:04:52.243486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1389/2774] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:04:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:04:58.668194 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=25 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:05:02.239475 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=6 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:05:30.236555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:05:30.238004 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:30.250429 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf9db-fc96-4738-b420-886eedfdf9a0 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:05:30.253947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:05:30.254577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:05:30.254858 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:05:30.255129 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:05:30.256846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:30.317468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:05:30.318916 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:05:30.328065 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1386/2775] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:05:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:05:40.323776 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=28 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:05:42.914369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:05:42.914966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:05:42.914966 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:05:42.932970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] MFA Rules not processed for user `f1d4360de70b4beea3456aa3dc85d5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:05:42.934988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:42.941847 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1390/2776] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:05:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:05:43.959498 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:05:43.961029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:44.033965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c850c-0eb2-4682-84b3-24c0084e2619 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:05:44.038908 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:05:44.039560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:05:44.039843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:05:44.040118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:05:44.041960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:44.053982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:05:44.055620 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:05:44.066351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1387/2777] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:05:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:05:47.710948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:05:47.713075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:47.732695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751497f4-90ee-4128-b1f0-d9430f84b1f5 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:05:47.738830 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:05:47.739493 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:05:47.739934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:05:47.740241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:05:47.742167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:05:47.756425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:05:47.758070 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:05:47.771449 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1391/2778] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:05:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:05:54.062007 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=32 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:05:57.765266 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=9 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:06:00.405387 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:06:00.419071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe4e58-173a-48d7-b53e-0e7f61f15503 service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:06:00.424441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:06:00.425104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:06:00.425410 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:06:00.425659 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:06:00.427416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:00.439660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:06:00.441270 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:06:00.452146 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1388/2779] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:06:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:06:03.973264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:06:03.974651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:03.985827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d42f3-dfbe-4a72-a90a-9cf129eaf89a service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:06:03.989801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:06:03.990428 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:06:03.990712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:06:03.990989 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:06:03.992699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:04.055100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:06:04.056887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:06:04.067178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1392/2780] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:06:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:06:05.951895 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:06:05.953436 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:05.969199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e00f-4069-4d73-b8af-8641c0ae29a4 service nova] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:06:05.975267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:06:05.976057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:06:05.976413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:06:05.976825 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:06:05.978774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:05.991937 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:06:05.993684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08cac7-ca74-44c2-a585-d6cd7a29f463 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:06:06.005964 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1389/2781] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:06:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 04:06:14.062590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=30 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:06:16.001700 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=12 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:06:33.124246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:06:33.140582 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01739b-73f4-4a84-b460-0839ccc00205 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:06:33.143458 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:06:33.144141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:06:33.144430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:06:33.144747 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:06:33.146559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:33.157331 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:06:33.158912 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:06:33.168178 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1393/2782] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:06:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:06:43.163730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=6 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:06:52.941779 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:06:52.942333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:06:52.942442 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:06:52.959860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] MFA Rules not processed for user `18d67c48b3a5445296a2069be87ebae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:06:52.961093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:52.968208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1390/2783] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:06:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:06:53.259941 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:06:53.269328 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c18e6d-e43c-448f-8d40-163c56f045ce service nova] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:06:53.272887 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:06:53.273524 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:06:53.273813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:06:53.274098 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:06:53.275844 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:06:53.285979 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:06:53.287393 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:06:53.295900 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1394/2784] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:06:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:07:02.954765 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:07:03.291749 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=6 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:09.695186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:09.701312 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:09.711328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92166e54-6fe6-4214-9d8d-2531cd5b5bf2 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:09.716927 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:09.717590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:09.717875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:09.718150 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:09.719880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:09.731882 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:09.733521 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:09.743597 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1391/2785] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:19.739226 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=6 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:20.663202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:20.672234 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235c6f5-0780-4444-8a35-5b606608e43d service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:20.676074 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:20.676739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:20.677027 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:20.677307 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:20.679095 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:20.742923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:20.744370 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:20.754400 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1395/2786] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:24.271069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:24.289529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcd00-4cc2-4ac9-94f5-8e3814d77bc4 service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:24.296195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:24.296910 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:24.297209 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:24.301742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:24.301742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:24.313529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:24.315130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:24.325759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1392/2787] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:27.862776 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:27.875813 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df768b9c-6940-4d98-97be-3f9449b5c187 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:27.881237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:27.881952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:27.882319 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:27.882650 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:27.884528 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:27.896893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:27.898468 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:27.909956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1396/2788] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:28.000035 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:28.001551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:28.015649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011c1c-347f-4e13-8221-e090705960d3 service placement] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:28.020163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:28.021184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:28.021461 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:28.021735 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] PATH_INFO: `/v3/users/f1d4360de70b4beea3456aa3dc85d5ec` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:28.022920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d4360de70b4beea3456aa3dc85d5ec)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:28.024877 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:28.220313 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d4360de70b4beea3456aa3dc85d5ec'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.221014 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d4360de70b4beea3456aa3dc85d5ec'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.232330 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d4360de70b4beea3456aa3dc85d5ec'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.233079 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d4360de70b4beea3456aa3dc85d5ec'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.324803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d4360de70b4beea3456aa3dc85d5ec'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.343114 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1393/2789] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:08:27 2022] DELETE /identity/v3/users/f1d4360de70b4beea3456aa3dc85d5ec => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:28.587295 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:28.599205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dc7cc-f4c7-41e2-9b06-9b6ca6abcb12 service placement] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:28.667000 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:28.667864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:28.668155 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:28.668439 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] PATH_INFO: `/v3/projects/5e8a7647a47e4ac19972582346dbdcee` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:28.669658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8a7647a47e4ac19972582346dbdcee)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:28.672160 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:28.790738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8a7647a47e4ac19972582346dbdcee', 'request_id': 'req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.879434 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8a7647a47e4ac19972582346dbdcee', 'request_id': 'req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.886064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8a7647a47e4ac19972582346dbdcee', 'request_id': 'req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.896461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] The token cache is being invalidate because project 5e8a7647a47e4ac19972582346dbdcee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:08:28.896781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.897411 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:28.898935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1397/2790] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:08:28 2022] DELETE /identity/v3/projects/5e8a7647a47e4ac19972582346dbdcee => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:28.903067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:28.904507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:28.918145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5e2ca-f16d-4bef-bf51-8530177e7d62 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:28.974715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:28.975638 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:28.975924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:28.976205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] PATH_INFO: `/v3/users/77d9fb0819ec4442bf7584680dbb4cb2` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:28.977416 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77d9fb0819ec4442bf7584680dbb4cb2)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:28.979359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:29.177195 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77d9fb0819ec4442bf7584680dbb4cb2'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.177851 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77d9fb0819ec4442bf7584680dbb4cb2'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.184621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77d9fb0819ec4442bf7584680dbb4cb2'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.185257 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77d9fb0819ec4442bf7584680dbb4cb2'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.189147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:29.307596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77d9fb0819ec4442bf7584680dbb4cb2'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.325738 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1394/2791] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:08:28 2022] DELETE /identity/v3/users/77d9fb0819ec4442bf7584680dbb4cb2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:29.581342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:29.584138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:29.598103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689ea9-c4dc-44c4-a019-c06946a1ecb0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:29.606361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:29.655061 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:29.656577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:29.656865 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:29.657141 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] PATH_INFO: `/v3/projects/a02d65db8b4d46b48a3dd131e59fe886` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:29.658320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02d65db8b4d46b48a3dd131e59fe886)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:29.660230 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:29.774187 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02d65db8b4d46b48a3dd131e59fe886', 'request_id': 'req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.830774 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02d65db8b4d46b48a3dd131e59fe886', 'request_id': 'req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.837183 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02d65db8b4d46b48a3dd131e59fe886', 'request_id': 'req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.847722 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] The token cache is being invalidate because project a02d65db8b4d46b48a3dd131e59fe886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:08:29.848011 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.848687 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:29.850272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1398/2792] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:08:29 2022] DELETE /identity/v3/projects/a02d65db8b4d46b48a3dd131e59fe886 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:39.373786 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:39.844733 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:42.961443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:42.962866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:43.019684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766241f-ce7a-4651-b276-cbda2ad9d582 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:43.067777 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:43.068433 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:43.068744 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:43.069025 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:43.070804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:43.131897 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:43.133294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:43.143281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1395/2793] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:46.470664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:46.472194 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:46.485790 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30b65e-8968-464a-b32e-bc4e9f81d1f3 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:46.490595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:46.491470 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:46.491762 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:46.492045 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] PATH_INFO: `/v3/users/18d67c48b3a5445296a2069be87ebae8` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:46.493253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d67c48b3a5445296a2069be87ebae8)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:46.495283 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:46.658075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d67c48b3a5445296a2069be87ebae8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:46.658708 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d67c48b3a5445296a2069be87ebae8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:46.659318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d67c48b3a5445296a2069be87ebae8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:46.671314 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d67c48b3a5445296a2069be87ebae8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:46.679249 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d682796f-4aac-4983-ab6e-0cf4770010bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d67c48b3a5445296a2069be87ebae8'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:46.743641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1399/2794] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:08:46 2022] DELETE /identity/v3/users/18d67c48b3a5445296a2069be87ebae8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:47.011491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.022039 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c34172-5d13-439b-9216-a89db551b497 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.078636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.079503 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.079791 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.080082 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] PATH_INFO: `/v3/projects/c80fc33b8e6b4d359f432ace6c297b80` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.081267 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80fc33b8e6b4d359f432ace6c297b80)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.083186 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.090949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=100 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:47.213337 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80fc33b8e6b4d359f432ace6c297b80', 'request_id': 'req-524ce08d-0415-4457-ae89-66844f1b1924'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:47.219349 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80fc33b8e6b4d359f432ace6c297b80', 'request_id': 'req-524ce08d-0415-4457-ae89-66844f1b1924'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:47.299527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80fc33b8e6b4d359f432ace6c297b80', 'request_id': 'req-524ce08d-0415-4457-ae89-66844f1b1924'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:47.311184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] The token cache is being invalidate because project c80fc33b8e6b4d359f432ace6c297b80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:08:47.311475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:47.312245 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-524ce08d-0415-4457-ae89-66844f1b1924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:47.314351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1396/2795] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:08:47 2022] DELETE /identity/v3/projects/c80fc33b8e6b4d359f432ace6c297b80 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:47.319517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.319816 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.320097 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.338022 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:47.339220 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:47.400286 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1400/2796] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:08:47.409058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.409355 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.409636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.428280 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:47.429527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:47.437369 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1397/2797] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:08:47.441431 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.450867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c596f-57b7-47f2-944e-8f838f3c2c47 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.456094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.456766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.457056 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.457335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.458058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:08:47.458367 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.459961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.472453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1401/2798] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:08:47.476590 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.486187 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c011a-243b-4433-b19c-0894add0f644 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.490917 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.491567 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.491853 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.492131 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.492860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.494772 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.580643 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1398/2799] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:08:47 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:08:47.584665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.595196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550622e-5a5a-435c-946f-e5ddaf62a0fb admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.651326 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.651988 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.652271 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.652576 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.653264 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.655161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.662397 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:47.756238 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1402/2800] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:08:47 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:08:47.760287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.771348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd318-4956-4f80-aa00-6198cf705b47 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.776055 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.776757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.777070 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.777353 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.778110 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.779748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.789854 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1399/2801] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:08:47.793759 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.804042 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b98190-02d2-4813-99be-f08dff3bf82e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.808559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dc546-0e36-4c68-953e-ce8aa117b2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.809207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc546-0e36-4c68-953e-ce8aa117b2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.809490 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc546-0e36-4c68-953e-ce8aa117b2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.809766 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dc546-0e36-4c68-953e-ce8aa117b2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.810480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dc546-0e36-4c68-953e-ce8aa117b2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.812062 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dc546-0e36-4c68-953e-ce8aa117b2a8 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.822365 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1403/2802] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 04:08:47.826069 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:47.836554 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afbb5b-64a5-430d-94a1-52602fc62864 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:47.841109 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:47.841975 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.842261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.842538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] PATH_INFO: `/v3/projects/be3f7ff22aa5428f86044acbc7795cc1/users/9cd17db7d3b1422182c3a52491eb680d/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.856708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3f7ff22aa5428f86044acbc7795cc1, user_id=9cd17db7d3b1422182c3a52491eb680d, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:47.859354 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:47.950630 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1400/2803] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:08:47 2022] PUT /identity/v3/projects/be3f7ff22aa5428f86044acbc7795cc1/users/9cd17db7d3b1422182c3a52491eb680d/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:47.962258 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:47.962552 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:47.962854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:47.988842 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] MFA Rules not processed for user `9cd17db7d3b1422182c3a52491eb680d`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:47.990043 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:48.052568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1404/2804] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:08:48.064594 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:48.066342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:48.138026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529202-487d-4e77-88e8-84440e694c9f admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:48.145522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf474e-e5cb-4d8e-9fd7-9a244be6b637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:48.146204 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf474e-e5cb-4d8e-9fd7-9a244be6b637 service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:48.146491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf474e-e5cb-4d8e-9fd7-9a244be6b637 service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:48.146769 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf474e-e5cb-4d8e-9fd7-9a244be6b637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:48.154684 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf474e-e5cb-4d8e-9fd7-9a244be6b637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:48.156272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf474e-e5cb-4d8e-9fd7-9a244be6b637 service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:48.168972 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1401/2805] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:57.262780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:57.322877 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fffb2-87ad-4d20-9145-e4fc11aaae7d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:57.326970 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:57.327845 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:57.328133 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:57.328461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] PATH_INFO: `/v3/users/9cd17db7d3b1422182c3a52491eb680d` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:57.330388 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd17db7d3b1422182c3a52491eb680d)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:57.332206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:57.567066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd17db7d3b1422182c3a52491eb680d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:57.567701 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd17db7d3b1422182c3a52491eb680d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:57.568293 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd17db7d3b1422182c3a52491eb680d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:57.581161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd17db7d3b1422182c3a52491eb680d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:57.585737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:57.588706 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd17db7d3b1422182c3a52491eb680d'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:57.692337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1405/2806] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:08:57 2022] DELETE /identity/v3/users/9cd17db7d3b1422182c3a52491eb680d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:57.696860 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:57.697350 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:57.697477 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:57.715348 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:57.716585 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:57.720610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:57.774276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1402/2807] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:08:57.784180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:57.785755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:57.842800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a40a80-13a6-4886-879d-05c1cba74a86 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:57.846442 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c01df-9b03-4090-a92a-053ef8ba4f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:57.847084 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c01df-9b03-4090-a92a-053ef8ba4f8b service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:57.847371 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c01df-9b03-4090-a92a-053ef8ba4f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:57.847649 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c01df-9b03-4090-a92a-053ef8ba4f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:57.856453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c01df-9b03-4090-a92a-053ef8ba4f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:57.857908 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c01df-9b03-4090-a92a-053ef8ba4f8b service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:57.867965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1406/2808] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:08:58.257248 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:58.258718 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:58.271759 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d696a3e-8019-4ab8-b5ec-8a838c77e652 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:58.276973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:58.277846 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.278128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.278401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] PATH_INFO: `/v3/projects/be3f7ff22aa5428f86044acbc7795cc1` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.279576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3f7ff22aa5428f86044acbc7795cc1)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:58.281529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:58.395158 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3f7ff22aa5428f86044acbc7795cc1', 'request_id': 'req-2fa22865-9b18-4e31-a085-582d518113a5'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:58.400413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3f7ff22aa5428f86044acbc7795cc1', 'request_id': 'req-2fa22865-9b18-4e31-a085-582d518113a5'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:58.477955 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3f7ff22aa5428f86044acbc7795cc1', 'request_id': 'req-2fa22865-9b18-4e31-a085-582d518113a5'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:58.489024 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] The token cache is being invalidate because project be3f7ff22aa5428f86044acbc7795cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:08:58.493532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:58.493532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-2fa22865-9b18-4e31-a085-582d518113a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:08:58.493532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1403/2809] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:08:58 2022] DELETE /identity/v3/projects/be3f7ff22aa5428f86044acbc7795cc1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:58.496559 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.497055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.497173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.515152 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:58.516340 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:58.575994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1407/2810] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:08:58.584978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.585277 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.585566 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.604488 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:58.605741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:58.613878 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1404/2811] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:08:58.618250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:58.627827 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcd062-64dd-4050-910f-4088c3a75b71 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:58.632626 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:58.633277 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.633562 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.633837 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.634566 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:08:58.634871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:58.636590 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:58.654081 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1408/2812] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:08:58.657163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:58.667586 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6483998-c8ad-475e-86f6-377f301d66bf None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:58.672266 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:58.672990 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.674049 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.674339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.675057 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:58.677188 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:58.789321 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1405/2813] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:08:58 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:08:58.793440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:58.803931 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e36ea-a9cc-4d43-ad15-360c92857e6c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:58.858870 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:58.859536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.859823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.860103 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.860895 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:58.862839 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:58.953408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1409/2814] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:08:58 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:08:58.959104 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:58.969528 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89d983-70a2-48a7-bd97-2aa546fe1ea1 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:58.974162 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:58.974812 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:58.975093 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:58.975364 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:58.976077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:58.977704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:58.988111 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1406/2815] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:08:58.991973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:59.002606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953aa4c-3a31-4b29-b955-32c28ebb5ad5 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:59.007272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89918b-83c4-48d6-83ac-d84c683b9b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:59.007925 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89918b-83c4-48d6-83ac-d84c683b9b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:59.008212 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89918b-83c4-48d6-83ac-d84c683b9b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:59.008523 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89918b-83c4-48d6-83ac-d84c683b9b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:59.009306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89918b-83c4-48d6-83ac-d84c683b9b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:59.010893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89918b-83c4-48d6-83ac-d84c683b9b3f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:59.012123 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:59.023437 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1410/2816] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:08:59.027467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:59.037559 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe9d40-26fa-4c6a-aa9d-b0ed01237247 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:59.042012 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:59.042902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:59.043189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:59.043467 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] PATH_INFO: `/v3/projects/635977b26bc148f6b88002d325b2e116/users/aa4811a153044da99f96aa8827c8dc03/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:59.058058 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=635977b26bc148f6b88002d325b2e116, user_id=aa4811a153044da99f96aa8827c8dc03, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:59.060793 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:59.164078 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1407/2817] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:08:59 2022] PUT /identity/v3/projects/635977b26bc148f6b88002d325b2e116/users/aa4811a153044da99f96aa8827c8dc03/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:08:59.175357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:59.176681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:59.176681 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:59.205320 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] MFA Rules not processed for user `aa4811a153044da99f96aa8827c8dc03`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:08:59.205684 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:59.271866 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1411/2818] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:08:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:08:59.283413 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:08:59.285191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:08:59.317757 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:08:59.358191 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd01a1-52c4-4fa7-8c89-00b1d001bea4 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:08:59.364476 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af7430-e86b-4d60-bd41-a383fe50c7bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ceb759c7b4b4292bb9763574d79254b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:08:59.365182 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af7430-e86b-4d60-bd41-a383fe50c7bf service nova] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:08:59.365468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af7430-e86b-4d60-bd41-a383fe50c7bf service nova] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:08:59.365743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af7430-e86b-4d60-bd41-a383fe50c7bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:08:59.373803 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af7430-e86b-4d60-bd41-a383fe50c7bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:08:59.375391 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af7430-e86b-4d60-bd41-a383fe50c7bf service nova] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:08:59.385076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1408/2819] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:08:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:02.670072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:02.732446 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa777ef7-719e-45b7-a87c-c015c29dc515 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:02.737208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:02.738085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:02.738374 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:02.738658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] PATH_INFO: `/v3/users/aa4811a153044da99f96aa8827c8dc03` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:02.739885 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4811a153044da99f96aa8827c8dc03)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:02.742170 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:02.974092 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4811a153044da99f96aa8827c8dc03'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:02.974750 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4811a153044da99f96aa8827c8dc03'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:02.975361 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4811a153044da99f96aa8827c8dc03'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:02.987207 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4811a153044da99f96aa8827c8dc03'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:02.993923 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4811a153044da99f96aa8827c8dc03'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:03.080786 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1412/2820] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:02 2022] DELETE /identity/v3/users/aa4811a153044da99f96aa8827c8dc03 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:09:03.085451 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:03.085922 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:03.086026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:03.104071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:03.105316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:03.163334 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1409/2821] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:03.173351 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:03.174800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:03.181325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:03.231517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef98cc6-2824-4798-ae7c-66b78bc80bac admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:03.235085 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10688537-a7c2-446a-950e-c2695b87a9b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:03.236316 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688537-a7c2-446a-950e-c2695b87a9b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:03.236630 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688537-a7c2-446a-950e-c2695b87a9b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:03.236914 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688537-a7c2-446a-950e-c2695b87a9b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:03.246305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10688537-a7c2-446a-950e-c2695b87a9b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:03.252347 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10688537-a7c2-446a-950e-c2695b87a9b5 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:03.257801 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1413/2822] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 04:09:03.719562 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:03.727171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:03.733560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d249d6a-b6f4-43cf-8a9a-a211734ac541 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:03.738876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:03.739745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:03.740029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:03.740310 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] PATH_INFO: `/v3/projects/635977b26bc148f6b88002d325b2e116` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:03.741536 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=635977b26bc148f6b88002d325b2e116)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:03.743468 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:03.874173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '635977b26bc148f6b88002d325b2e116', 'request_id': 'req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:03.880795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '635977b26bc148f6b88002d325b2e116', 'request_id': 'req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:03.946660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '635977b26bc148f6b88002d325b2e116', 'request_id': 'req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:03.958376 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] The token cache is being invalidate because project 635977b26bc148f6b88002d325b2e116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80701) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:03.958660 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:03.959278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a88b5211-9cfb-4a0d-9fe3-a9639d2a5c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:03.960834 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1410/2823] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:03 2022] DELETE /identity/v3/projects/635977b26bc148f6b88002d325b2e116 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:03.967715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:03.967715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:03.967715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:03.985013 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:03.986315 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:04.045190 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1414/2824] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:04.054362 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.054649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.054921 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.073022 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:04.074219 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:04.082611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1411/2825] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:04.086597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.096994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5b98a-6608-49a7-9d4e-d20c6fd5ad43 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.101572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.102223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.102513 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.102797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] PATH_INFO: `/v3/domains` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.103527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80700) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:09:04.103838 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.105463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.116919 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1415/2826] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:09:04.120965 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.130268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f94c88-2d37-438b-bbaf-e6e7b21b938b None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.134818 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.135459 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.135741 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.136015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.136726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.138621 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.212171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1412/2827] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:04 2022] POST /identity/v3/projects => generated 405 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:04.216203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.226881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5888a-1a73-4920-b016-5797b2dc9532 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.282255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.282927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.283274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.283668 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.284373 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.286333 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.350453 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1416/2828] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:04 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:04.354624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.365669 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e6897-a93c-4c6b-acb5-0df8bf68018e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.370432 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.371084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.371366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.371641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.372359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.374027 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.376043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:04.384845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1413/2829] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:04.388755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.399369 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958a805-00f1-4e74-a44c-a99f2ca2a6d7 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.404058 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8618fa-e8a6-49f8-952a-45735a55af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.404811 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8618fa-e8a6-49f8-952a-45735a55af4d admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.405201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8618fa-e8a6-49f8-952a-45735a55af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.405539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8618fa-e8a6-49f8-952a-45735a55af4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.406270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8618fa-e8a6-49f8-952a-45735a55af4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.407868 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8618fa-e8a6-49f8-952a-45735a55af4d admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.418854 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1417/2830] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:04.423288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.433752 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde9e9-2869-45f1-bacb-ade37db1c4a6 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.438583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.439616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.439904 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.440184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] PATH_INFO: `/v3/projects/49f7cbe777fe4cd28f3e9455a0bf2c1b/users/413d16c4a221449292f82466e9d245d6/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.455052 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f7cbe777fe4cd28f3e9455a0bf2c1b, user_id=413d16c4a221449292f82466e9d245d6, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.457830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.543773 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1414/2831] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:04 2022] PUT /identity/v3/projects/49f7cbe777fe4cd28f3e9455a0bf2c1b/users/413d16c4a221449292f82466e9d245d6/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:04.554344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.554641 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.554927 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.578102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:04.581623 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] MFA Rules not processed for user `413d16c4a221449292f82466e9d245d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:04.582869 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:04.646248 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1418/2832] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:04.650774 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.715934 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee895c76-dfa7-4cbc-9c98-7d2eeb58ec24 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.720636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.721907 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.722196 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.722473 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] PATH_INFO: `/v3/projects` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.723165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.725098 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:04.854430 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1415/2833] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:04 2022] POST /identity/v3/projects => generated 405 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:04.858499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:04.869539 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68528f21-a1f4-4b11-a0e9-e05bc7c8c2c1 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:04.927440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:04.928111 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:04.928402 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:04.928718 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] PATH_INFO: `/v3/users` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:04.929419 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:04.931337 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.000991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1419/2834] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:04 2022] POST /identity/v3/users => generated 309 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:05.005787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:05.016743 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6759cf8-1aad-4ba4-9dc8-25427459046c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:05.022551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:05.023243 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.023527 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.023804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.024547 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:05.026149 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.036892 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1416/2835] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:05.041376 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:05.051620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb4439-7be0-4ae6-8760-1659d8e46538 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:05.056033 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:05.056956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.057226 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.057502 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] PATH_INFO: `/v3/projects/2337c5cf2dc747aa8a34a18265052a13/users/fd2ee51214e8479e968490b16132e49d/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.072287 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2337c5cf2dc747aa8a34a18265052a13, user_id=fd2ee51214e8479e968490b16132e49d, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:05.075083 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.175282 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1420/2836] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:05 2022] PUT /identity/v3/projects/2337c5cf2dc747aa8a34a18265052a13/users/fd2ee51214e8479e968490b16132e49d/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:05.179199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:05.190198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85720d6b-5b17-475d-85c5-6e384388f3ad admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:05.245325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b8a79-10bd-436c-a3ec-87ac85a9d409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:05.245993 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8a79-10bd-436c-a3ec-87ac85a9d409 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.246285 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8a79-10bd-436c-a3ec-87ac85a9d409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.246551 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b8a79-10bd-436c-a3ec-87ac85a9d409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.247296 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b8a79-10bd-436c-a3ec-87ac85a9d409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:05.248932 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b8a79-10bd-436c-a3ec-87ac85a9d409 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.259046 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1417/2837] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:05.263267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:05.274736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3a6d5-af1b-433f-90e8-23377216ce59 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:05.279754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:05.280662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.280952 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.281233 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] PATH_INFO: `/v3/projects/2337c5cf2dc747aa8a34a18265052a13/users/fd2ee51214e8479e968490b16132e49d/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.283586 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2337c5cf2dc747aa8a34a18265052a13, user_id=fd2ee51214e8479e968490b16132e49d, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:05.286247 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.296243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1421/2838] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:05 2022] PUT /identity/v3/projects/2337c5cf2dc747aa8a34a18265052a13/users/fd2ee51214e8479e968490b16132e49d/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:05.307307 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.307600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.307879 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.334061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] MFA Rules not processed for user `fd2ee51214e8479e968490b16132e49d`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:05.335254 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:05.367292 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:05.401806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1418/2839] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:05.414114 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:05.415794 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:05.472822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6339f-cb1b-4882-b003-a9616020dc47 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:05.476261 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d0772-bdaa-4e94-9dca-84684b66d114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:05.476955 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d0772-bdaa-4e94-9dca-84684b66d114 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.477237 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d0772-bdaa-4e94-9dca-84684b66d114 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.477515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d0772-bdaa-4e94-9dca-84684b66d114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.486802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d0772-bdaa-4e94-9dca-84684b66d114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:05.488232 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d0772-bdaa-4e94-9dca-84684b66d114 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.492053 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:05.497913 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1422/2840] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:05 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:05.631118 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:05.693647 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d0235-3294-4c0f-b9d0-83e679cada11 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:05.698124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:05.698989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.699278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.699560 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] PATH_INFO: `/v3/users/413d16c4a221449292f82466e9d245d6` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:05.700771 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=413d16c4a221449292f82466e9d245d6)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:05.702703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:05.874273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413d16c4a221449292f82466e9d245d6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:05.874963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413d16c4a221449292f82466e9d245d6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:05.883701 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413d16c4a221449292f82466e9d245d6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:05.884384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413d16c4a221449292f82466e9d245d6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:05.960375 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413d16c4a221449292f82466e9d245d6'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:05.978333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1419/2841] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:05 2022] DELETE /identity/v3/users/413d16c4a221449292f82466e9d245d6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:05.982871 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:05.983290 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:05.983589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:06.002496 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:06.006438 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:06.062654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1423/2842] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:06.073041 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:06.074466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:06.128583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:06.134142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cbfac-5873-4e9f-8076-ba0f4eaf8432 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:06.137680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:06.138325 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:06.138609 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:06.138883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:06.148240 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:06.149715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:06.159696 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1420/2843] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:06.548942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:06.550462 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:06.563253 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa2046-5732-4192-b6fc-364c1a2707a0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:06.568867 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:06.569741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:06.570023 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:06.570299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] PATH_INFO: `/v3/projects/49f7cbe777fe4cd28f3e9455a0bf2c1b` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:06.571527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f7cbe777fe4cd28f3e9455a0bf2c1b)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:06.573504 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:06.693059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f7cbe777fe4cd28f3e9455a0bf2c1b', 'request_id': 'req-44fa6c48-8d77-4339-b67c-9ec6072aae57'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:06.758272 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f7cbe777fe4cd28f3e9455a0bf2c1b', 'request_id': 'req-44fa6c48-8d77-4339-b67c-9ec6072aae57'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:06.764782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f7cbe777fe4cd28f3e9455a0bf2c1b', 'request_id': 'req-44fa6c48-8d77-4339-b67c-9ec6072aae57'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:06.775709 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] The token cache is being invalidate because project 49f7cbe777fe4cd28f3e9455a0bf2c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:06.775998 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:06.776672 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:06.778203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1424/2844] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:06 2022] DELETE /identity/v3/projects/49f7cbe777fe4cd28f3e9455a0bf2c1b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:06.782316 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:06.792095 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13b021-5cd9-49b4-b42b-7e871c567ca3 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:06.849218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:06.850085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:06.850372 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:06.850649 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] PATH_INFO: `/v3/users/fd2ee51214e8479e968490b16132e49d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:06.851821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2ee51214e8479e968490b16132e49d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:06.853794 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:07.034518 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2ee51214e8479e968490b16132e49d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.035269 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2ee51214e8479e968490b16132e49d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.042740 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2ee51214e8479e968490b16132e49d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.043368 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2ee51214e8479e968490b16132e49d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.106512 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-07621f8b-108e-4fff-9dc0-82ce9bc658d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2ee51214e8479e968490b16132e49d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.124576 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1421/2845] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:06 2022] DELETE /identity/v3/users/fd2ee51214e8479e968490b16132e49d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:07.562161 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:07.563799 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:07.580627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa6c48-8d77-4339-b67c-9ec6072aae57 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:07.640089 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:07.640992 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:07.641284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:07.641569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] PATH_INFO: `/v3/projects/2337c5cf2dc747aa8a34a18265052a13` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:07.643356 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2337c5cf2dc747aa8a34a18265052a13)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:07.645399 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:07.670422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:07.743055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2337c5cf2dc747aa8a34a18265052a13', 'request_id': 'req-f66c0677-6f48-40ca-be11-8ec38fcb596a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.809631 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2337c5cf2dc747aa8a34a18265052a13', 'request_id': 'req-f66c0677-6f48-40ca-be11-8ec38fcb596a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.816107 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2337c5cf2dc747aa8a34a18265052a13', 'request_id': 'req-f66c0677-6f48-40ca-be11-8ec38fcb596a'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.826461 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] The token cache is being invalidate because project 2337c5cf2dc747aa8a34a18265052a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:07.826735 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.827363 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f66c0677-6f48-40ca-be11-8ec38fcb596a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:07.828934 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1425/2846] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:07 2022] DELETE /identity/v3/projects/2337c5cf2dc747aa8a34a18265052a13 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:07.834359 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:07.836208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:07.836208 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:07.853710 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:07.854998 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:07.914177 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1422/2847] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:07.923614 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:07.924068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:07.924713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:07.943529 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:07.949999 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:07.953985 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1426/2848] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:07.958924 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:07.968709 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e6e1-654c-4b19-bf51-378b9535db08 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:07.973600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:07.974252 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:07.974535 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:07.974810 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:07.975538 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:09:07.975847 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:07.977552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:07.984145 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:07.989278 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1423/2849] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 04:09:07.993604 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.003443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51749d0-f9c8-416a-8213-2b66752edd5d None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.008112 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.008810 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.009100 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.009380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.010080 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.011976 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.124910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1427/2850] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:07 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:09:08.129600 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.140171 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12387a1-e74b-4599-8659-43536c95023e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.195702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.196412 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.196727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.197008 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.197694 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.199613 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.272797 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1424/2851] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:08 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 04:09:08.276849 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.288191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca8803-aee4-4981-976e-37e4af8a85b5 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.292792 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.293488 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.293782 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.294069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.294822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.296422 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.307864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1428/2852] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:08.311654 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.322273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32b22f-238b-49b6-80f0-cbe0ced002fa admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.326889 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e2370-b980-4a2b-92b6-bd9daf8d1d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.327540 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2370-b980-4a2b-92b6-bd9daf8d1d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.327829 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2370-b980-4a2b-92b6-bd9daf8d1d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.328142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2370-b980-4a2b-92b6-bd9daf8d1d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.328902 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e2370-b980-4a2b-92b6-bd9daf8d1d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.330571 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e2370-b980-4a2b-92b6-bd9daf8d1d65 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.340389 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1425/2853] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:08.344306 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.354359 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac432a80-db27-4ed7-b778-2def42beb63f admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.358910 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.359780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.360068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.360342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] PATH_INFO: `/v3/projects/979a9680a36241c391bd5fef49430805/users/b8fe6b0c7c8240838a701eeb02e1e5ad/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.374994 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=979a9680a36241c391bd5fef49430805, user_id=b8fe6b0c7c8240838a701eeb02e1e5ad, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.378201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.484006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1429/2854] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:08 2022] PUT /identity/v3/projects/979a9680a36241c391bd5fef49430805/users/b8fe6b0c7c8240838a701eeb02e1e5ad/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:08.494733 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.495026 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.495328 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.522804 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] MFA Rules not processed for user `b8fe6b0c7c8240838a701eeb02e1e5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:08.524002 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:08.588085 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1426/2855] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:08.592517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.655771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccf5d2-f1aa-47c9-b997-ce6bcd062934 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.661797 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.662465 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.662755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.663059 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.663754 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.665710 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.762148 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1430/2856] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:08 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:08.765997 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.777036 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5f80e-1069-48ac-bbd1-da1d50c38eb4 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.833680 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.834343 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.834624 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.835173 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.835578 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.837517 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.847425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:08.960840 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1427/2857] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:08 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:08.965294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:08.977939 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5c895-ba2e-4416-9b7e-e2e9b4b09a52 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:08.982492 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:08.986846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:08.986846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:08.986846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:08.986846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:08.986846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:08.998656 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1431/2858] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:09.002668 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.013176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3a826-f5c8-4055-a391-9c38f817d70e admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.018616 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ea1e7-788a-4fba-8e1d-a39feedf29bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.019304 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ea1e7-788a-4fba-8e1d-a39feedf29bc admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.019589 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ea1e7-788a-4fba-8e1d-a39feedf29bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.019920 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ea1e7-788a-4fba-8e1d-a39feedf29bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.020682 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ea1e7-788a-4fba-8e1d-a39feedf29bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.022333 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ea1e7-788a-4fba-8e1d-a39feedf29bc admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.032979 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1428/2859] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:09.037620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.047627 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6303e1c-6633-48db-9498-5e704ec90781 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.052713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.053591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.053881 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.054163 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] PATH_INFO: `/v3/projects/94ee134375b34fdfb7556a5a7155ca84/users/21e5eeb5eb5c4b4d8b900af56ddf6417/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.068741 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ee134375b34fdfb7556a5a7155ca84, user_id=21e5eeb5eb5c4b4d8b900af56ddf6417, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.071380 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.160935 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:09.166430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1432/2860] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:09 2022] PUT /identity/v3/projects/94ee134375b34fdfb7556a5a7155ca84/users/21e5eeb5eb5c4b4d8b900af56ddf6417/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:09.177982 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.178205 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.178514 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.205849 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] MFA Rules not processed for user `21e5eeb5eb5c4b4d8b900af56ddf6417`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:09.207089 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:09.270443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1429/2861] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:09.274897 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.336719 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280661a9-24c1-4171-8604-ebcbd16dbbbc admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.341303 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.341956 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.342245 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.342527 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.343223 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.345159 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.425752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1433/2862] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:09 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:09.429801 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.440596 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb7ff0-dd1d-4b04-9796-cb3c0f678917 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.498329 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.498670 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.498986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.499273 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.499976 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.502124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.577787 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1430/2863] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:09 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:09.582184 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.592587 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5845ea2-4595-4e9d-b552-49157192fcbb admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.598460 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.599173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.599600 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.599888 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.600658 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.602299 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.614243 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1434/2864] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:09.618274 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.629137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f165b9-6988-461c-a114-3e16e27d4faf admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.633967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.634844 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.635130 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.635411 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] PATH_INFO: `/v3/projects/2c6e3e219161483d9d5fb01e0f657e35/users/236168f39a78410ab4f6ac5c75578011/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.650627 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6e3e219161483d9d5fb01e0f657e35, user_id=236168f39a78410ab4f6ac5c75578011, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.653339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.717674 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1431/2865] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:09 2022] PUT /identity/v3/projects/2c6e3e219161483d9d5fb01e0f657e35/users/236168f39a78410ab4f6ac5c75578011/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:09.722175 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.732156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bed42-3eb7-4b01-92cb-4b8b0274b648 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.788573 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93690f6-39fc-4ec7-9a2a-2788b6eff6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.789231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93690f6-39fc-4ec7-9a2a-2788b6eff6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.789519 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93690f6-39fc-4ec7-9a2a-2788b6eff6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.789795 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93690f6-39fc-4ec7-9a2a-2788b6eff6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.790515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93690f6-39fc-4ec7-9a2a-2788b6eff6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.792101 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93690f6-39fc-4ec7-9a2a-2788b6eff6ab admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.802034 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1435/2866] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:09.806471 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.819529 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f2140-a7da-42b8-92ce-626557d29c19 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:09.824239 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:09.825143 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.825425 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.825702 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] PATH_INFO: `/v3/projects/2c6e3e219161483d9d5fb01e0f657e35/users/236168f39a78410ab4f6ac5c75578011/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.827671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6e3e219161483d9d5fb01e0f657e35, user_id=236168f39a78410ab4f6ac5c75578011, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:09.831009 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:09.841142 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1432/2867] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:09 2022] PUT /identity/v3/projects/2c6e3e219161483d9d5fb01e0f657e35/users/236168f39a78410ab4f6ac5c75578011/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:09.852765 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:09.853064 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:09.853349 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:09.879463 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] MFA Rules not processed for user `236168f39a78410ab4f6ac5c75578011`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:09.880711 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:09.937497 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:09.947699 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1436/2868] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:09.959361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:09.961075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:09.993117 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:10.019253 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aaab4d-f7d6-4616-b08e-16d0ad4ed55b admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:10.022896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:10.027122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:10.027122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:10.027122 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:10.033845 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:10.035276 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:10.044584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1433/2869] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:10.180149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:10.181598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:10.194652 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84744c-64b4-49cb-8592-8ad574ec76a0 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:10.198188 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac8e8d-07ac-4376-82b8-9d385653902f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:10.198826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac8e8d-07ac-4376-82b8-9d385653902f service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:10.199168 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac8e8d-07ac-4376-82b8-9d385653902f service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:10.199451 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac8e8d-07ac-4376-82b8-9d385653902f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:10.256321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac8e8d-07ac-4376-82b8-9d385653902f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:10.257726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac8e8d-07ac-4376-82b8-9d385653902f service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:10.266691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1437/2870] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:14.215272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:14.276061 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ad5bb-c22c-4b6f-84c1-2b3617aac2a5 service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:14.280544 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:14.281400 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:14.281681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:14.281957 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] PATH_INFO: `/v3/users/b8fe6b0c7c8240838a701eeb02e1e5ad` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:14.283218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8fe6b0c7c8240838a701eeb02e1e5ad)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:14.285475 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:14.422288 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8fe6b0c7c8240838a701eeb02e1e5ad'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:14.423715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8fe6b0c7c8240838a701eeb02e1e5ad'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:14.431071 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8fe6b0c7c8240838a701eeb02e1e5ad'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:14.431704 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8fe6b0c7c8240838a701eeb02e1e5ad'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:14.483045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8fe6b0c7c8240838a701eeb02e1e5ad'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:14.500938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1434/2871] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:14 2022] DELETE /identity/v3/users/b8fe6b0c7c8240838a701eeb02e1e5ad => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:14.506425 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:14.506949 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:14.507055 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:14.524096 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:14.525321 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:14.585577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1438/2872] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:09:14.595383 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:14.596978 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:14.652464 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:14.656335 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf5877-eb17-45c1-bac9-496cddff2e69 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:14.660689 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:14.661332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:14.661611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:14.661883 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:14.670723 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:14.672156 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:14.682736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1435/2873] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:15.087826 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:15.089350 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:15.102231 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544ba0-96aa-41a1-8ced-721311be3af7 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:15.107595 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:15.107771 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:15.108068 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:15.108348 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] PATH_INFO: `/v3/projects/979a9680a36241c391bd5fef49430805` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:15.109599 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=979a9680a36241c391bd5fef49430805)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:15.111597 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:15.212007 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979a9680a36241c391bd5fef49430805', 'request_id': 'req-249e601e-8e0a-4c9b-8f0d-201724d3a436'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.283953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979a9680a36241c391bd5fef49430805', 'request_id': 'req-249e601e-8e0a-4c9b-8f0d-201724d3a436'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.290554 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979a9680a36241c391bd5fef49430805', 'request_id': 'req-249e601e-8e0a-4c9b-8f0d-201724d3a436'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.301121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] The token cache is being invalidate because project 979a9680a36241c391bd5fef49430805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:15.301413 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.302032 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.303560 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1439/2874] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:15 2022] DELETE /identity/v3/projects/979a9680a36241c391bd5fef49430805 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:15.307507 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:15.317176 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79ba9-985a-408e-9f8c-830f258df7e3 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:15.372933 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:15.373795 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:15.374077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:15.374351 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] PATH_INFO: `/v3/users/21e5eeb5eb5c4b4d8b900af56ddf6417` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:15.375532 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e5eeb5eb5c4b4d8b900af56ddf6417)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:15.377681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:15.566553 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e5eeb5eb5c4b4d8b900af56ddf6417'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.567128 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e5eeb5eb5c4b4d8b900af56ddf6417'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.573966 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e5eeb5eb5c4b4d8b900af56ddf6417'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.574584 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e5eeb5eb5c4b4d8b900af56ddf6417'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.657179 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e5eeb5eb5c4b4d8b900af56ddf6417'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:15.674951 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1436/2875] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:15 2022] DELETE /identity/v3/users/21e5eeb5eb5c4b4d8b900af56ddf6417 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:16.087415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:16.088972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:16.102517 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e601e-8e0a-4c9b-8f0d-201724d3a436 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:16.120689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:16.165683 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:16.166577 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:16.166861 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:16.167140 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] PATH_INFO: `/v3/projects/94ee134375b34fdfb7556a5a7155ca84` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:16.168334 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ee134375b34fdfb7556a5a7155ca84)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:16.170323 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:16.279556 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ee134375b34fdfb7556a5a7155ca84', 'request_id': 'req-f3ca5b9e-e642-444e-a253-a8c45a912bf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.377541 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ee134375b34fdfb7556a5a7155ca84', 'request_id': 'req-f3ca5b9e-e642-444e-a253-a8c45a912bf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.384416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ee134375b34fdfb7556a5a7155ca84', 'request_id': 'req-f3ca5b9e-e642-444e-a253-a8c45a912bf1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.395006 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] The token cache is being invalidate because project 94ee134375b34fdfb7556a5a7155ca84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:16.395294 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.395987 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.397665 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1440/2876] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:16 2022] DELETE /identity/v3/projects/94ee134375b34fdfb7556a5a7155ca84 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:16.401637 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:16.403044 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:16.416341 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c75fa6-98a0-4079-83ab-c25af1133fad admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:16.472673 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:16.473583 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:16.473866 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:16.474139 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] PATH_INFO: `/v3/users/236168f39a78410ab4f6ac5c75578011` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:16.475315 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236168f39a78410ab4f6ac5c75578011)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:16.477284 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:16.489815 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:16.671165 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236168f39a78410ab4f6ac5c75578011'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.671821 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236168f39a78410ab4f6ac5c75578011'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.678736 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236168f39a78410ab4f6ac5c75578011'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.679361 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236168f39a78410ab4f6ac5c75578011'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.742960 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-246d60a8-4242-4b57-81d1-70d1cc92c6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236168f39a78410ab4f6ac5c75578011'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:16.761483 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1437/2877] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:16 2022] DELETE /identity/v3/users/236168f39a78410ab4f6ac5c75578011 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:17.208360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:17.209928 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:17.222893 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5b9e-e642-444e-a253-a8c45a912bf1 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:17.280596 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:17.281454 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.281739 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.282016 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] PATH_INFO: `/v3/projects/2c6e3e219161483d9d5fb01e0f657e35` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.283205 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6e3e219161483d9d5fb01e0f657e35)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:17.285169 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:17.429606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:17.431274 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6e3e219161483d9d5fb01e0f657e35', 'request_id': 'req-05c6abf6-e312-4d2b-b9bd-8fe12c259786'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:17.519964 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6e3e219161483d9d5fb01e0f657e35', 'request_id': 'req-05c6abf6-e312-4d2b-b9bd-8fe12c259786'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:17.526585 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6e3e219161483d9d5fb01e0f657e35', 'request_id': 'req-05c6abf6-e312-4d2b-b9bd-8fe12c259786'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:17.537196 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] The token cache is being invalidate because project 2c6e3e219161483d9d5fb01e0f657e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:17.537489 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:17.538135 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-05c6abf6-e312-4d2b-b9bd-8fe12c259786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:17.539737 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1441/2878] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:17 2022] DELETE /identity/v3/projects/2c6e3e219161483d9d5fb01e0f657e35 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:09:17.550067 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.550587 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.550726 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.569539 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:17.571096 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:17.632522 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1438/2879] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:17.641051 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.641408 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.641712 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.661131 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:17.662357 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:17.670485 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1442/2880] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:17.675763 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:17.685339 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44c73d-f488-4fc5-b642-58391f88d310 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:17.689822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:17.690472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.690755 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.691029 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.691745 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:09:17.692053 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:17.693715 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:17.705297 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1439/2881] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:09:17.709639 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:17.718953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585f7d0-19a8-4425-940c-b0027fb03f6b None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:17.724172 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:17.724852 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.725138 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.725415 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.726102 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:17.727990 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:17.822620 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1443/2882] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:17 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:17.826401 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:17.836294 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f44871-ac9d-42b1-8a62-235bdfc0b010 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:17.895875 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:17.896568 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.896857 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.897137 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.897828 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:17.899748 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:17.971681 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1440/2883] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:17 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:17.975654 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:17.986713 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84024c-dc35-4c5d-bbd8-699db871d1a0 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:17.991412 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:17.992075 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:17.992354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:17.992742 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:17.993534 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:17.995130 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:18.005745 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1444/2884] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:18.009491 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:18.019953 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4915c2-452a-4414-aa50-e1913302915a admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:18.024598 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f1044-600b-4e7a-b9fb-b526a88bed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:18.025256 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1044-600b-4e7a-b9fb-b526a88bed9a admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:18.025541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1044-600b-4e7a-b9fb-b526a88bed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:18.025822 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f1044-600b-4e7a-b9fb-b526a88bed9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:18.026541 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f1044-600b-4e7a-b9fb-b526a88bed9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:18.028229 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f1044-600b-4e7a-b9fb-b526a88bed9a admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:18.029561 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:18.040019 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1441/2885] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:18.044121 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:18.054416 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88806c5d-0375-4f30-8abc-f32a60c7f84e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:18.058991 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:18.059864 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:18.060149 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:18.060430 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] PATH_INFO: `/v3/projects/e6002017bbf3404381e391e6a32275c8/users/f7b2037a2eec4bcbb665b256f04f929d/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:18.078781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6002017bbf3404381e391e6a32275c8, user_id=f7b2037a2eec4bcbb665b256f04f929d, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:18.078781 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:18.169608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1445/2886] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:18 2022] PUT /identity/v3/projects/e6002017bbf3404381e391e6a32275c8/users/f7b2037a2eec4bcbb665b256f04f929d/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:18.174880 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf72a-286f-4409-9f9c-d4fd868191fe None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:18.175167 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf72a-286f-4409-9f9c-d4fd868191fe None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:18.175443 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf72a-286f-4409-9f9c-d4fd868191fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:18.192614 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-3dcdf72a-286f-4409-9f9c-d4fd868191fe None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:18.193896 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3dcdf72a-286f-4409-9f9c-d4fd868191fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:18.250942 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1442/2887] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:18.260467 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:18.262202 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:18.320977 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a68910-03c8-47e6-9b0d-336a32773236 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:18.324606 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:18.325256 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:18.325544 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:18.325824 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:18.335863 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:18.337267 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:18.347726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1446/2888] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:19.545977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:19.546452 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:19.546577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:19.575817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] MFA Rules not processed for user `f7b2037a2eec4bcbb665b256f04f929d`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:19.577086 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:19.645371 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1443/2889] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:19.650948 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:19.661255 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ca927-fefe-4336-b2e4-fd2e03eda8d6 service neutron] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:19.666823 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:19.667486 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:19.667778 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:19.668127 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:19.668821 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:19.670760 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:19.755942 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1447/2890] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:19 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:19.759999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:19.771448 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fa9b0-7500-4461-b69c-457c33e17d70 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:19.825798 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:19.826388 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:19.826671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:19.826947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:19.827640 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:19.830261 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:19.892076 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1444/2891] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:19 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:19.895953 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:19.906572 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d4da3-0efb-4168-8f75-eb673020574b admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:19.911963 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:19.912674 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:19.912962 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:19.913241 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:19.914019 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:19.915756 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:19.927360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1448/2892] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:19.931360 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:19.941336 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7fe4f-3b84-4c2e-9915-6d683cee4dd2 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:19.945843 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:19.946708 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:19.946989 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:19.947263 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] PATH_INFO: `/v3/projects/3b8d096e79ea44199e650cdedbdc98d6/users/3c20cc0d5eee4fa2b4c4828c70c37bef/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:19.958963 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:19.961985 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8d096e79ea44199e650cdedbdc98d6, user_id=3c20cc0d5eee4fa2b4c4828c70c37bef, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:19.964732 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:20.014742 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1445/2893] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:19 2022] PUT /identity/v3/projects/3b8d096e79ea44199e650cdedbdc98d6/users/3c20cc0d5eee4fa2b4c4828c70c37bef/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:09:20.019280 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:20.029203 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a85ee-c1b0-49ac-ab1d-f276c7e9f47e admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:20.069480 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:20.086066 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc68c-fa8e-419b-853c-a58e757c3422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:20.086675 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc68c-fa8e-419b-853c-a58e757c3422 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:20.086965 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc68c-fa8e-419b-853c-a58e757c3422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:20.087250 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc68c-fa8e-419b-853c-a58e757c3422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:20.087975 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc68c-fa8e-419b-853c-a58e757c3422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:20.089642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc68c-fa8e-419b-853c-a58e757c3422 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:20.099836 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1449/2894] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:20.104077 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:20.114268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894f53-9c77-4d08-8c15-1aff68f9fcbf admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:20.118817 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:20.119683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:20.119968 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:20.120247 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] PATH_INFO: `/v3/projects/3b8d096e79ea44199e650cdedbdc98d6/users/3c20cc0d5eee4fa2b4c4828c70c37bef/roles/7842553ae25f43ec83b0bb89e9fc9175` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:20.122235 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8d096e79ea44199e650cdedbdc98d6, user_id=3c20cc0d5eee4fa2b4c4828c70c37bef, role_id=7842553ae25f43ec83b0bb89e9fc9175)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:20.125084 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:20.135967 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1446/2895] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:20 2022] PUT /identity/v3/projects/3b8d096e79ea44199e650cdedbdc98d6/users/3c20cc0d5eee4fa2b4c4828c70c37bef/roles/7842553ae25f43ec83b0bb89e9fc9175 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:21.475857 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:21.476344 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:21.476598 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:21.504443 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] MFA Rules not processed for user `3c20cc0d5eee4fa2b4c4828c70c37bef`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:21.505695 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:21.577972 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1450/2896] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:21.591890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:21.593664 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:21.651397 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc96b05-34ed-43e3-9121-17f4e5758621 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:21.655028 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:21.655671 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:21.655949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:21.656224 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:21.713075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:21.714511 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:21.723287 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1447/2897] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:23.457409 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:23.458800 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:23.470856 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478bae-7ab7-4f95-bffe-4947f8120214 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:23.474574 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:23.475224 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:23.475538 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:23.475822 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:23.486228 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:23.487653 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:23.496980 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1451/2898] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:31.719147 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:33.492904 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=50 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:39.183407 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:39.249342 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6138fd4-b0f9-4e34-aae9-513e2468d35e service cinder] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:39.249775 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:39.250641 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:39.250928 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:39.251202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] PATH_INFO: `/v3/users/f7b2037a2eec4bcbb665b256f04f929d` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:39.252409 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b2037a2eec4bcbb665b256f04f929d)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:39.254363 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:39.439537 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b2037a2eec4bcbb665b256f04f929d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:39.440198 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b2037a2eec4bcbb665b256f04f929d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:39.448283 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b2037a2eec4bcbb665b256f04f929d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:39.448944 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b2037a2eec4bcbb665b256f04f929d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:39.528446 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b2037a2eec4bcbb665b256f04f929d'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:39.546228 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1448/2899] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:39 2022] DELETE /identity/v3/users/f7b2037a2eec4bcbb665b256f04f929d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:39.789501 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:39.800386 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0564d8-fc3e-4b01-9533-95b845f767a8 service cinder] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:39.857780 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:39.858642 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:39.858929 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:39.859206 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] PATH_INFO: `/v3/projects/e6002017bbf3404381e391e6a32275c8` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:39.860377 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6002017bbf3404381e391e6a32275c8)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:39.862396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:39.959536 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6002017bbf3404381e391e6a32275c8', 'request_id': 'req-32fbb439-c136-40cf-ab50-16aa9b407210'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.019846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6002017bbf3404381e391e6a32275c8', 'request_id': 'req-32fbb439-c136-40cf-ab50-16aa9b407210'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.026803 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6002017bbf3404381e391e6a32275c8', 'request_id': 'req-32fbb439-c136-40cf-ab50-16aa9b407210'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.037843 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] The token cache is being invalidate because project e6002017bbf3404381e391e6a32275c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:40.038069 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.038736 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.040284 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1452/2900] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:39 2022] DELETE /identity/v3/projects/e6002017bbf3404381e391e6a32275c8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:40.044300 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:40.045754 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:40.059610 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dc3d7-b8d3-4e7a-8649-88baf0846112 admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:40.115500 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:40.116366 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:40.116683 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:40.116959 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] PATH_INFO: `/v3/users/3c20cc0d5eee4fa2b4c4828c70c37bef` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:40.118125 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c20cc0d5eee4fa2b4c4828c70c37bef)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:40.120051 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:40.141190 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:40.293075 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c20cc0d5eee4fa2b4c4828c70c37bef'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.293703 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c20cc0d5eee4fa2b4c4828c70c37bef'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.300645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c20cc0d5eee4fa2b4c4828c70c37bef'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.301281 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c20cc0d5eee4fa2b4c4828c70c37bef'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.341629 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-5d0da912-f092-4230-a672-c81c3422097c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c20cc0d5eee4fa2b4c4828c70c37bef'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.359482 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1449/2901] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:40 2022] DELETE /identity/v3/users/3c20cc0d5eee4fa2b4c4828c70c37bef => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:09:40.635983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:40.637229 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:40.651730 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbb439-c136-40cf-ab50-16aa9b407210 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:40.712589 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:40.713464 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:40.713752 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:40.714036 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] PATH_INFO: `/v3/projects/3b8d096e79ea44199e650cdedbdc98d6` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:40.715227 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8d096e79ea44199e650cdedbdc98d6)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:40.717201 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:40.815920 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:40.827335 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8d096e79ea44199e650cdedbdc98d6', 'request_id': 'req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.925167 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8d096e79ea44199e650cdedbdc98d6', 'request_id': 'req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.933691 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8d096e79ea44199e650cdedbdc98d6', 'request_id': 'req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.945260 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] The token cache is being invalidate because project 3b8d096e79ea44199e650cdedbdc98d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:40.945551 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.946246 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-c07be588-4e0f-4ab6-bdc4-36f2af4629a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:40.947802 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1453/2902] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:40 2022] DELETE /identity/v3/projects/3b8d096e79ea44199e650cdedbdc98d6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 04:09:40.952973 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:40.953268 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:40.953555 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:40.971766 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:40.973087 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:41.033552 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1450/2903] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:41.041983 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.042270 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.042643 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.061896 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:41.064179 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:41.071755 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1454/2904] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:41.075947 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.085727 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36762220-100f-46f1-9a8b-caeecb28af59 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.090332 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.090986 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.091271 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.091548 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.092272 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80701) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 04:09:41.092611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.094234 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.106206 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1451/2905] 10.222.0.58 () {58 vars in 1152 bytes} [Fri May 27 04:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 04:09:41.110329 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.119860 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ddcb5-2ab5-4b2c-9eb9-cb38d6a30937 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.124396 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.125072 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.125354 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.125628 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.126308 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.128173 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.218608 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1455/2906] 10.222.0.58 () {60 vars in 1152 bytes} [Fri May 27 04:09:41 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:41.222753 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.233871 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb7964-8c3a-424c-9b24-1f60d8988daa admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.289652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.290322 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.290602 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.290876 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] PATH_INFO: `/v3/users` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.291550 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.293580 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.409199 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1452/2907] 10.222.0.58 () {60 vars in 1143 bytes} [Fri May 27 04:09:41 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 04:09:41.414065 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.424785 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e300c4b-83c8-4289-8455-eb6f5cc83b07 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.429715 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.430360 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.430651 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.430937 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.431662 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.433305 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.444054 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1456/2908] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 04:09:41.447890 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.457806 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091daa-8ab9-40f4-81d4-a7ca2198937c admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.462961 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6c2b1-8972-4e66-a096-010d07d0f196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.463617 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6c2b1-8972-4e66-a096-010d07d0f196 admin admin] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.463905 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6c2b1-8972-4e66-a096-010d07d0f196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.464184 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6c2b1-8972-4e66-a096-010d07d0f196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.464977 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6c2b1-8972-4e66-a096-010d07d0f196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.466577 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6c2b1-8972-4e66-a096-010d07d0f196 admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.477231 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1453/2909] 10.222.0.58 () {58 vars in 1121 bytes} [Fri May 27 04:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 04:09:41.481208 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.491106 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63d0bf-8060-43e4-a3c4-a7afae1b4f88 admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.495664 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.496591 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] REQUEST_METHOD: `PUT` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.496875 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.497156 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] PATH_INFO: `/v3/projects/11f9635be91545be93c5c2a755b4bba5/users/77e410e541d84dc08ab5b61039aacd98/roles/93828c8fe28c41ca831b85d55b1a50ea` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.511689 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f9635be91545be93c5c2a755b4bba5, user_id=77e410e541d84dc08ab5b61039aacd98, role_id=93828c8fe28c41ca831b85d55b1a50ea)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.514342 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.634738 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1457/2910] 10.222.0.58 () {60 vars in 1482 bytes} [Fri May 27 04:09:41 2022] PUT /identity/v3/projects/11f9635be91545be93c5c2a755b4bba5/users/77e410e541d84dc08ab5b61039aacd98/roles/93828c8fe28c41ca831b85d55b1a50ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:41.646306 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] REQUEST_METHOD: `POST` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.647163 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.647449 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.654178 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:41.674836 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.auth.core [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] MFA Rules not processed for user `77e410e541d84dc08ab5b61039aacd98`. Rule list: `[]` (Enabled: `True`). {{(pid=80701) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:41.676045 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:41.743189 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1454/2911] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 04:09:41.754353 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.756569 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:41.814878 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a0d14-e569-49fd-8bd9-a906ebb6e64c admin admin] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.819440 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd0935-19e5-4aad-be50-7ed943e0f411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd4fb121b8de406899d5698c84d0903b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.820094 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0935-19e5-4aad-be50-7ed943e0f411 service cinder] REQUEST_METHOD: `GET` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.820382 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0935-19e5-4aad-be50-7ed943e0f411 service cinder] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.820690 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd0935-19e5-4aad-be50-7ed943e0f411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.828921 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd0935-19e5-4aad-be50-7ed943e0f411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.830345 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd0935-19e5-4aad-be50-7ed943e0f411 service cinder] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:41.840185 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1458/2912] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:41.864193 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:41.926151 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286559f-45ad-4095-9ea3-8a120a8d8f61 None None] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:41.931202 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:41.932063 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:41.932346 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:41.932652 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] PATH_INFO: `/v3/users/77e410e541d84dc08ab5b61039aacd98` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:41.933867 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e410e541d84dc08ab5b61039aacd98)` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:41.935785 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:42.081830 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e410e541d84dc08ab5b61039aacd98'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.082509 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e410e541d84dc08ab5b61039aacd98'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.089472 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e410e541d84dc08ab5b61039aacd98'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.090124 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e410e541d84dc08ab5b61039aacd98'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.148015 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.notifications [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e410e541d84dc08ab5b61039aacd98'} {{(pid=80701) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.166758 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1455/2913] 10.222.0.58 () {58 vars in 1223 bytes} [Fri May 27 04:09:41 2022] DELETE /identity/v3/users/77e410e541d84dc08ab5b61039aacd98 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:42.171602 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] REQUEST_METHOD: `POST` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:42.171958 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:42.172325 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:42.189930 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.auth.core [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] MFA Rules not processed for user `e836482fa8a8442295d2819d77066796`. Rule list: `[]` (Enabled: `True`). {{(pid=80700) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 04:09:42.191180 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:42.241726 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:42.249905 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1459/2914] 10.222.0.58 () {58 vars in 957 bytes} [Fri May 27 04:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 04:09:42.260121 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:42.268384 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:42.275949 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:42.319023 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d4cb-d9ef-434f-91eb-fe4c110ef9fe admin admin] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:42.322636 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65d7683f531a41b1adf3ceb94d32ce2c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:42.323331 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:42.323611 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:42.323881 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:42.332678 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:42.334213 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:42.343466 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1456/2915] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:09:42.772555 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] Authenticating user token {{(pid=80700) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:09:42.773961 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.fernet_utils [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80700) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:09:42.785740 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d489660-9e00-4f73-96f2-238950cbfc51 None None] Validating token access rules against request {{(pid=80700) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:09:42.791313 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e836482fa8a8442295d2819d77066796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1781b8350b784207b421bac2bc80c855', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80700) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:09:42.792257 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:09:42.792568 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:09:42.792846 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] PATH_INFO: `/v3/projects/11f9635be91545be93c5c2a755b4bba5` {{(pid=80700) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:09:42.794191 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f9635be91545be93c5c2a755b4bba5)` {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:09:42.796622 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] RBAC: Authorization granted {{(pid=80700) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:09:42.920151 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f9635be91545be93c5c2a755b4bba5', 'request_id': 'req-d88d23a6-8268-415b-9b59-45d5bc60d950'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.987515 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f9635be91545be93c5c2a755b4bba5', 'request_id': 'req-d88d23a6-8268-415b-9b59-45d5bc60d950'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:42.994525 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f9635be91545be93c5c2a755b4bba5', 'request_id': 'req-d88d23a6-8268-415b-9b59-45d5bc60d950'} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:43.005026 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] The token cache is being invalidate because project 11f9635be91545be93c5c2a755b4bba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80700) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 04:09:43.005318 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:43.006499 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG keystone.notifications [None req-d88d23a6-8268-415b-9b59-45d5bc60d950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80700) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 04:09:43.008109 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: [pid: 80700|app: 0|req: 1460/2916] 10.222.0.58 () {58 vars in 1232 bytes} [Fri May 27 04:09:42 2022] DELETE /identity/v3/projects/11f9635be91545be93c5c2a755b4bba5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 04:09:52.476814 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=20 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:09:53.001973 dv-cinder-843361-2-iscsi devstack@keystone.service[80700]: DEBUG dbcounter [-] [80700] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80700) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 04:13:44.895441 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] Authenticating user token {{(pid=80701) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 04:13:44.896874 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:13:45.042073 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77022d5f-b4fe-4907-84e3-42c7706d8ea0 service neutron] Validating token access rules against request {{(pid=80701) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 04:13:45.100999 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f611173abbd4a89b4f762571a760a27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c2c9c8b9edfd489d9c51c633882a4cc7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80701) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 04:13:45.101651 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] REQUEST_METHOD: `GET` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 04:13:45.101938 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] SCRIPT_NAME: `/identity` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 04:13:45.102218 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80701) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 04:13:45.104043 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.fernet_utils [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80701) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 04:13:45.115645 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 04:13:45.117286 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2ab31-5408-4625-a810-a2455dee935b service placement] RBAC: Authorization granted {{(pid=80701) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 04:13:45.127103 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: [pid: 80701|app: 0|req: 1457/2917] 10.222.0.58 () {60 vars in 1365 bytes} [Fri May 27 04:13:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 04:13:55.122783 dv-cinder-843361-2-iscsi devstack@keystone.service[80701]: DEBUG dbcounter [-] [80701] Writing DB stats keystone:SELECT=89 {{(pid=80701) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}